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Abstract 

This article presents the major findings from a meta-analysis of six regional analyses conducted 
within the framework of the European Commission Web-edu project 

(http://www.nettskolen.com/in english/webedusite/index.htmi y It analyses the satisfaction, or 
lack of satisfaction, of European institutions with the e-Leaming Learning Management 
Systems (LMSs) that they have purchased or developed themselves. Data was collected from 
in-depth interviews with 113 European experts, usually the e-Leaming systems managers in the 
institutions, in 17 countries. The analyses of the interviews revealed as many as 52 different 
commercial and 35 self-developed LMS systems. The article presents the data from these 
interviews and includes a series of far-reaching conclusions from the study. A striking 
conclusion of this study is that the generally accepted position that the market is dominated by 
the American LMSs, is not the norm throughout Europe. In the countries that not use English 
as their first language, locally developed LMS systems have successfully repelled the American 
products. A remarkable large number of the LMS systems used in Europe are commercial 
systems developed locally or self-developed systems at the institutions. 



Introduction 

There is now little doubt that the World Wide Web is the most successful educational tool to 
have appeared in a long time. It combines and integrates text, audio and video with interaction 
amongst participants. It can be used on a global scale and is platform independent. While 
largely an asynchronous medium, it can also be used for synchronous events. It is not 
surprising therefore, that trainers, lecturers, distance education providers, and teaching 
institutions at all levels are increasingly using the World Wide Web as a medium for course 
provision (Mason 1998). 



Education and training on the World Wide Web is generally referred to as e-Leaming. In the 
last few years it has spread worldwide and is now a recognised sector of education and training 
provision. It is an electronic form of distance learning and is available, in addition, to on- 
campus students who can use e- Leaming provision to supplement their lectures and sessions in 
the computing laboratory. 

Today e-Leaming is widely used in corporate training and international and multinational 
corporations claim to have made vast savings in training costs by switching much of their 
training to the WWW. Nationally and internationally recognised university degrees, college 
diplomas and training certification are now available by e-Leaming. E-Leaming companies, like 
Riverdeep and SmartForce are quoted on the New York Nasdaq Exchange, and the major 
providers of Learning Management Systems, like WebCT and Blackboard, are major US 
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corporations. Few universities in the English-speaking world do not provide e-Leaming 
courses and the penetration of this provision into Europe is a growing phenomenon. 



Evidence of the world wide spread of e-Leaming in recent years is easy to obtain. No fewer 
than 70.000 courses are listed on the TeleCampus portal from TeleEducation, New Brunswick, 
Canada ('http://www.courses.telecampus.edu) with URLs for each course that makes it easy for 
prospective students to study course summaries with a view to enrolling. In spite of the 
competence of the TeleEducation database, most of the courses are from the United States and 
Canada, and it is unlikely that many, or any, of the provision of the 113 institutions studied in 
this report is listed by their portal. 



^ Subject Calegories * ^ TeleCampus Online Course Directorv - Nelscdpe 










...in the TeleCampus Online Course Directory, you1t find thousands of courses 
from around the world, available over the internet 



login 



Sign rne up! 



Forgot pas.9vwd? 



Becoming a registered 
user ofthe TeleCampus 
Course Directory allows 
users to take advantage 
of additional features. 






Advanced Sasrch 



Free Courses 



Programs 



Lar«guages 



U stings 



Institution Listings 



Subject Categories 

Top » 

Over 60.000 courses exist within the following subject categories... 



1 . Applied Sciences, ^10 

Technology 

2. Arts «11 

3. Business, Economics «13 

4. Education 4^10 

5. fieatth and Medicine «10 



6. Personal Interest, Leisure, -»^10 
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7. Sciences 4 . 1 8 

6. Social Sciences >21 

S. Vocational Training +2A 
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To find a course, you may browse the subject headings by clicking the subject 
category or heading. There are many sub categories. If you view a plus sign at the 
end of a subject heading or subcategory, more subcategories exist forthat 
discipline. Courses will be displayed In alphabetical order as you browse 
subjects. 






























Much of the world wide success of e-Leaming can be attributed to the availability of Learning 
Management systems (LMSs), also known as Virtual Learning Environments (VLEs) or 
learning platforms. An LMS enables an institution to develop electronic learning materials for 
students, to offer these courses electronically to students, to test and evaluate the students 
electronically, and to generate electronically student databases in which student results and 
progress can be charted. 



The authoritative Observatory on Borderless Higher Education (2002) comments: 

Recent years have witnessed remarkable growth in use of learning platforms in higher 
education around the world. Following longstanding computerisation in research and 
administration, learning platforms have brought information technology closer to the 
core of the higher education experience- teaching and learning. 
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Learning platforms offer enhanced student access to learning materials, straightforward 
integration of digital content and a range of student interaction and tracking services. 
Different systems have different emphases, but common features include content 
authoring tools, calendars, syllabi, discussion boards and assessment mechanisms. 



Terminology in this territory is often loose and poorly defined. Learning management systems, 
content management systems, learning platforms and portals are just some of the most common 
terms used to describe a range of teaching, learning and administrative software. The lines 
between learning platforms and other administrative software are increasingly blurred, as vendors 
seek to provide comprehensive solutions, either alone or in partnership with specialist firms. 



Learning platforms have also blurred the lines between campus-based higher education and 
distance learning, offering campus-based students the flexibility to access provision outside the 
traditional classroom. Many leading learning platforms grew out of small-scale developments in 
universities. Some higher education institutions continue to develop in-house systems or buy into 
open source alternatives, but an ever-larger majority are purchasing licenses for proprietary 
platforms. Indeed, two vendors. Blackboard and WebCT currently dominate the market, not only 
in their native North America, but internationally. Yet both have been trading for little more than 
five years. 

Brandon-hall.com defines an LMS as: "software that automates the administration of training 
events. An LMS registers users, tracks courses in a catalogue, and records data from learners; 
it also provides reports to management. An LMS is typically designed to handle courses by 
multiple publishers and multiple providers." 



The focus of the European Commission Web-edu project is, therefore, on the satisfaction, or 
lack of satisfaction, that European institutions have with the LMSs that they have purchased or 
developed themselves. This is a timely analysis because in the English-speaking world the e- 
Leaming industry is dominated by the major American LMS providers. This is in spite of the 
fact that a number of these, like Web CT, was developed by Murray Goldberg at the University 
of British Columbia in Vancouver, Canada and then sold to an American company in 
Pennsylvania, whereas TopClass originated as a European Commission project at University 
College Dublin, in Ireland, before becoming an Irish campus company and then migrating to 
the United States. 

A striking conclusion of this study is that this generally accepted position that the market is 
dominated by the major American LMSs, is not the norm throughout Europe. In countries that 
not use English as their first language, locally developed LMSs, in native languages, have 
successfully repelled the American products. 

The Web-edu project regards e-Leaming as an electronic form of distance learning and 
therefore concentrates on those sub-systems of a viable learning environment that characterise 
distance learning: 

• Course development tools 

• Student support tools 

• Tutor support tools 

• Administration 

• Technology 

• Financial issues. 



Data presentation 

This article represents a meta-analysis of six regional analyses conducted within the framework 
of the European Commission Web-edu project 
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('htt p://www.nettskolen.com/in_english/webedusite/index.htmiy It is based on the regional 
analyses listed in Table 1 . 



Table 1. List of regional analyses 



Regions- • v ''s;:- ' 


References to phonal analyses ::: 1 


:NutnbeT:.of r ' 
institutions 


1 

1 

L 


North Western Europe 


Keegan 2002 


18 


The Nordic Countries 


Paulsen 2002 


20 


Norwegian Universities and Colleges 


Runnesto and Ristesund 2002 


24 


Germany 


Fritsch and Fdllmer 2002 


17 


Southern Europe 


Dias, Dias, and Pimenta 2002 


20 


The Czech Republic and Slovakia 


MiSincova 2002 


14 











All regional analyses are based on in-depth interviews with systems managers or systems 
experts at the user institutions. The interviews were conducted in the Fall of 2001 and Spring 
of 2002 as face-to-face meetings, telephone interviews, or e-mail interviews. All interviews 
were based on a common interview guide. 



The researchers had no intentions of selecting interviewees that constituted a representative 
selection of European system managers for LMS systems. Data is provided for Norway, which 
virtually includes all the universities and colleges in that country, but it was not a goal of the 
project that every European country be included in the project or that every institution in a 
country could be covered. However, it is considered that the total of 113 institutions and the 
spread to 17 European countries gives an adequate database for far-reaching conclusions on 
the satisfaction of European institutions with the LMSs they have developed or purchased. 



The researchers were encouraged to find interviewees in various tjqjes of institutions. The 
experiences showed that it could be hard work to find interviewees that both were competent 
and willing to take the necessary time to participate. 
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Table 2. The types of institutions. Sorted by total number of institutions 



Type of institution 


u 


1 

■ -I 


•'••E:-' \ I:' 

. r 2 . : ^ E. 5 

..O. j u . 

: ■ r- -C/:: 


•V' 

O' ■ 


' 1 

' 1 
■JS 

'■ 2. - rf .• 

1 III '• 


. j 


Universities 


4 


g 


10 


8 


7 




r 


38 


Colleges of higher and further 
education 


7 








2 


20 


29 


Private companies 




1 


3 


2 


6 




12 


Distance education 
institutions 




3 




2 


1 


3 


9 


Non-Profit Institutions 
(training) 






6 


1 


1 




8 


Institutes of Technology 


5 












5 


Primary and secondary 
schools 




3 










3 


University centres 




1 




1 






2 


Training organizations 


1 


1 










2 


Consortia 




2 










2 


Government training agencies 


1 












1 


Commercial provider of LMS 
related services 




1 










1 


Anonymous 






1 








1 



Touii 






20 



20-j 



-;?:y u-f y: ■■■:{? i.^ 



•24': •.'....■..in::! 



Table 2 lists the types of institutions in the study. Many of the institutions are universities, 
institutes of technology and colleges of further and higher education. Other types of institutions 
are more or less underrepresented. This reflects the willingness of university or company e- 
Leaming systems managers to co-operate in the study. 



Data is tabulated from the study for the following: 







List of regional analyses (Table 1) 

Types of institutions in the study (Table 2) 

List of 17 countries in the study with demographical data (Table 3) 
Percentage of institutions with more than 50 online courses (Table 4) 
Distribution of institutions per number of courses (Table 5) 
Commercial LMSs in the analyses (Table 6) 

Self-developed systems (Table 7) 



Important themes that emerged during the analyses and developed in the article are: 



• Penetration of internet use and LMS systems 

• Large scale providers 

• Commercial LMS systems 

• Regional preferences and market leaders 

• Vendor specific issues 

• Competitive issues 

• Self-developed systems 

• LMS functionality 
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• Knowledge, policy, and strategy 

• E-leaming standards 

• Important jBndings from a European perspective. 

Penetration of Internet Use and LMS Systems 

Table 3 shows that 113 institutions from 17 countries in Europe were interviewed and gives 
the official languages, population and Internet penetration in the countries concerned. All of 
these are factors in the choice and selection of an LMS for e-Leaming. The data presented are 
compiled from various sources used in the regional analyses. The primary sources are CIA 
World Factbook 2002 and Eurostat 2002. 

From this analysis we would like to emphasise the differences between the Internet use in 
Northern and Southern Europe. The Internet users range from 50% in the Nordic countries to 
33% in North Western Europe, 30% in Germany, 18% in Southern Europe, and 10% in the 
Czech Republic. 



Table 3. List of Countries in analyses, Official Languages, Inhabitants and Internet 
penetration. Sorted by Internet Users per 100 inhabitants. 



Gouiury | ^ of. ■ j .Langnage.::;. 

1 insuauioiis . | Jikm;:; 

! imcrvicwed.-: i :. H:: 


[-Inliabiranis in . i Imerncii: | 

: jniliions i hosis per ‘ | 

: LOOinhj. 1 


iilhiernei | 
;iusefs-p€r;:;7: 

; 


Sweden 


5 


Swedish 


449 964 


8 : 9 ^ 


770 


56.4 


Norway 


28 


Norwegian 


324 220 


4.5 


11.2 


52.7 


Finland 


4 


Finnish 


337 030 


5.2 


13.6 


44.5 


Denmark 


5 


Danish 


43 094 


5.3 


13 


43.0 


Great Britain 


6 


English 


227 480 


57.6 




33.5 


Northern 

Ireland 


4 


English 


14 120 


1.6 




33.5 


Germany 


17 


German 


357 021 


82.2 


2.3 


29.6 


Ireland 


8 


English 


70 280 


3.8 


2.3 


27.5 


Italy 


6 


Italian 


301 230 


57.8 


2.7 


23.3 


France 


4 


French 


547 030 


59.5 


1.7 


16.9 


Switzerland 


1 


German 


41 290 


7.2 


4.4 


24.0 


Spain 


1 


Spanish 


504 782 


39.5 


1.4 


13.9 


Slovakia 


4 


Slovak 


48 845 


5.4 


0.7 


12.1 


Portugal 


8 


Portuguese 


92 391 


10.2 


1.2 


10.0 


Czech Republic 


10 


Czech 


78 866 


10.3 


1.6 


9.7 


Greece 


1 


Greek 


131 940 


10.6 


1.0 


9.5 


Iceland 


1 


Icelandic 










Totaiiof IT 
countries ' 




Tatal-bf 14 ■■ ■ 
languasEs:-: -I 













There are significant regional differences within Europe with regard to how far the institutions 
have come in their use of LMS systems. The differences seem to follow the regional statistics 
for Internet users, which means that the Southern Europe, the Czech Republic and Slovakia 
seems to be less mature with regard to use of LMS systems than the other regions. 

The analyses for North Western Europe and the Nordic countries shows that these regions 
already have come far in their use of LMS systems: 



LMS systems seem to be widely used in Nordic higher, further, and continuing education. It 
is not easy to find Nordic institutions without experiences with LMS systems. (Paulsen 
2002) 
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In the United Kingdom and Ireland there is a very extensive implementation of e-Leaming 
via LMSs. This includes provision at degree and diploma level. It seems that very many 
universities and colleges have purchased an LMS, and many corporations too. (Keegan 
2002) 

The analyses for Southern Europe, the Czech Republic and Slovakia include the following 
statements, which indicate that these regions are less mature: 

From the present research it is clear that the increasing number of Internet users in Southern 
Europe is pushing up the e-Leaming market. There are more institutions, which have web 
presence and e-leaming offerings. (Dias, Dias, and Pimenta 2002) 





It is our conclusion from the present studies that Southern European institutions are on the 
right track to further develop their existing e-Leaming offerings. (Dias, Dias, and Pimenta 
2002) 

Since the year 1998 we have observed the e-leaming field evolution. Ana Dias (2000) wrote 
in the CISAER final report: The present study shows evidences of an evolution of the 
institutions involved in e-Leaming in Southern Europe. The pilot projects are no longer 
dominating the e-Leaming field in Southern Europe. But, the research also led us to observe 
that the most part of the e-Leaming managers assume a position of experimentation and 
initiation on the e-Leaming process (50% of the researched institutions have less then 15 
courses online). (Dias, Dias, and Pimenta 2002) 

The thrust in e-leaming is not yet wide spread. General public opinion about online 
education is not always positive. The institutions are many times viewed as the ones 
implementing strange “things”. In most cases online education is used as a help and addition 
to the traditional face-to-face education. However there are individual experiments and, as 
one institution stated, they would like to adjust their LMS in the way that it enables them to 
offer paid courses as a kind of lifelong education to the public. (Micincova 2002) 

9 from 14 interviewed institutions have not been using LMS longer then one year. That 
shows the very early start but the results after such a short time are already visible. The 
acceleration of this development must be considered. (Micincova 2002) 



In the Czech republic recently a nice co-operation in a project of building a virtual university 
has started among 3 universities (SBA in Karvina, University of Ostrava and FE). 
(Micincova 2002) 
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Large-scale providers of e-learning 

An important issue is the number of institutions that can be characterised as large-scale 
providers of e-Leaming and those in which provision is, as yet, on a smaller basis: 

The analysis indicates that there is a clear trend towards large-scale online education in the 
Nordic countries. It shows that 12 of the 20 institutions offer at least 50 online courses. 
According to a 1998-99 analysis, (Paulsen 2000) only 3 of 22 surveyed Nordic institutions 
offered more than 50 online courses three years ago. Further, the interviewees talk about 
LMS systems as large-scale systems capable of handling thousands of users. (Paulsen 2002) 

The first data analyses on large-scale providers, in which the provision of 50 or more courses 
online is considered to represent large-scale provision, is presented in Table 4. 



Table 4. Percentage of institutions with more than 50 online courses sorted by 
percentage 



Redoa 


Reference to regictial analyses ; 


iaiicasf-SOQnJmc ■ 
: courses 


i; Percetua^ ; : 


the Nordic Countries 


Paulsen 2002^ 


12 out of 20 


60 


Germany 


Fritsch and Fdllmer 2002 


7 out of 17 


41 


North Western Europe 


Keegan 2002 


6 out of 18 


33 


Southern Europe 


Dias, Dias, and Pimenta 2002 


5 out of 20 


25 


The Czech Republic and 
Slovakia 


Micincova 2002 


0 out of 14 


0 


Total ' :i 










If one characterizes institutions that offer at least 50 online courses as large-scale providers of 
online education, 30 of the 89 institutions (34%) we have data from could be characterized as 
large-scale providers. The analyses indicate that the trend towards large-scale online education 
has come further in the Nordic countries (60%) than in the other regions. 



Table 5. Distribution of institutions per number of courses 



Number of i 
courses ■’ ! 


\ The Nordic . 
: Coururies' ’ ’ 


i.Gerntany . : North . 

; Western 
; Europe 


Souihcm : 
Europe’ 


The Czech. | Sum [.CiSAER 

Refk^lic and | Web-ebu 1 anali'sis 
•Slovakia ■ : analyses- i (Paulsen 

■ f. I200C-V ' 




■■I 






ill 


MIS 


ii»i 








i 










1 


0 


6 


0 


0 


6 


1 


lb 


2 


7 


1 


4 


4 


23 


30 


2-4 


5 


1 


18 


3 


11 


2 


10 


2 


14 


2 


11 


10 


23 


30 


5-15 


10 


2 


24 


4 


22 


4 


33 


7 


21 


3 


22 


20 


22 


28 


16-99 


35 


7 


35 


6 


17 


3 


38 


8 


36 


5 


32 


29 


22 


28 


100- 


40 


8 


24 


4 


22 


4 


0 


0 


0 


0 


18 


16 


3 


4 


No Answer 


10 


2 


0 


0 


22 


4 


10 


2 


21 


3 


12 


11 


8 


10 


Totall : lOO 


, 20i 10(1 : 17< lOOi 18 


: iOiii n 






I lot ; : 330 i 



Table 5 compares the number of online courses found in this analysis with a previous 
international analysis of web-based education conducted in the CISAER-project (Paulsen 
2000). This comparison indicates that there is a clear trend that institutions offer more online 
courses today than they did three years ago. One may say that the trend is to go from small- 
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scale to large-scale online education. 



Commercial LMS Systems 

Table 6 shows that the analyses revealed that the 113 institutions altogether had experiences 
with 52 different commercial systems. It is however important to observe that only a few 
systems are used by several institutions. The analyses indicates that the following systems are 
among the most used commercial LMS systems in Europe, since they were the only systems 
that five or more institutions had experiences with: 



BlackBoard (14 institutions) 

WebCT (20 institutions) 

FirstClass (7 institutions) 

TopClass (7 institutions) 

Lotus Learning Space (6 institutions) 
ClassFronter (16 institutions) 

LUVIT (5 institutions) 

Tutor2000 (5 institutions) 



The strong position of the two North American systems BlackBoard and WebCT is not 
surprising, since they presently are the two dominant systems on the international market 
{Observatory on Borderless Higher Education, 2002): 



Some higher education institutions continue to develop in-house systems or buy into 
open source alternatives, but an ever-larger majority are purchasing licenses for 
proprietary platforms. Indeed, two vendors. Blackboard and WebCT currently dominate 
the market, not only in their native North America, but internationally. Yet both have 
been trading for little more than five years. Market consolidation is also underway. 

FirstClass is a Canadian system that seems to have a strong position in Scandinavia, and Lotus 
Learning Space is an IBM product that is also much used in Europe. 



The analyses found four European LMS systems seem to be significant competitors on the 
European market. TopClass may have a strong position in Europe since it originated in Ireland. 
ClassFronter is a Norwegian developed system that has a very dominant position in Norwegian 
universities and colleges. The system is available in a number of languages and sold to 
institutions in several countries. LUVIT originated at the University of Lund in Sweden, before 
it became a Swedish commercial company with reasonable success in Scandinavia and some 
other countries. Tutor2000 seems to be a successful LMS provider in the Czech Republic. 



Table 6 lists the 52 commercial LMSs identified in the study with their provenance, URL and 
extent of usage: 
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Table 6. The Commercial LMS systems included in the analyses 



Co-mnsfciaf 

systems. 



OrigLnaf 

JJationaliWii 



: imL efIMS 



I Sum of 



• V i#Insr, • 

■ I using ii us I using ii 3S j inst. 



Ascot CourseMaster 


Great British 


www.ascot-svstems.co.uk 


1 


6 


r 


Aspen 


American 


www.click2Ieam.com 


I 


I 


2 


Aulanet 






0 


I 


I 


BettyCOM 


Swedish 




0 


I 


2 


BlackBoard 


American 


www.blackboard.com 


9 


5 


14 


Centra 


American 


www.centra.com 


I 


0 


I 


ClassFronter 


Norwegian 


www.fronter.com 


16 


0 


16 


Clix campus 


German 


httn://camousonline.uni- 

freiburg.de:8181 


I 


0 


I 


COM-C 


Danish 


www.comc.dk 


0 


I 


I 


Corporate learning 


German 


WWW. global-learning, de 


I 


0 


I 


CourseKeeper 


Norwegian 


www.coursekeeper.com 


2 


0 


2 


Decus System 






0 


I 


I 


Destinations 






0 


I 


I 


DLS from ETS 


German 


www.click2a-online.com 


I 


0 


I 


Docent 


American 


www.docent.com 


I 


3 


4 


EDWIN 


Danish 




0 


I 


I 


FDL Learning 
Environment 


Great British 


httD://le.reading-colleee.ac.uk 


I 


0 


I 


FirstClass 


Canadian 


www.firstclass. com 


3 


4 


7 


Fle3 


Finish 


http://fle3 .uiah.fi/ 


I 


0 


I 


GLN- Global 
Learning Network 


American 


http://cisco.netacad.net 


I 


0 


I 


Granada Leamwise 


Great British 


www.oakwise.oakland.ac.uk 


I 


0 


I 


Imaker 


Norwegian 


www.imaker.no 


0 


I 


I 


Interwise-ecp 


German 


www.leamnetz-sh.de 


I 


0 


I 


Intraleam 


American 


WWW. intraleam. com 


2 


I 


3 


Intranets 


American 


WWW. intranets, com 


I 


I 


2 


IT Campus 






I 


0 


I 


It’s Learning 


Norwegian 


www.itsolutions.no 


I 


0 


I 


Kark 


Norwegian 


http://kark.uib.no 


I 


0 


I 


LC Profiler 


Finnish 


www.lcprof.com 


I 


0 


I 


Learning solution 


German 




I 


0 


I 


Leamlink evoeye 


American 


WWW. leamlink. com 


I 


0 


I 


LEKTOR 


Swedish 




0 


I 


I 


Lotus Learning Space 


American 


www.lotus.com 


3 


3 


6 


LUVIT 


Swedish 


www.luvit.com 


5 


0 


5 


Nettutor 






0 


I 


I 


Ping Pong 


Swedish 


www.partitur.se 


I 


0 


I 


Plato 






0 


I 


I 


Proto 






0 


I 


I 


Response 








I 


I 


Saba 


American 


www.saba.com 


I 


2 


3 


Simulnet 






0 


I 


I 


Skills Vantage 






0 


I 


I 


Solstra Hybrid 






0 


I 


I 


TeamWave 








I 


I 


TopClass 


Irish 


www.wbtsvstems.com 


5 


2 


7 


Tutor2000 


Czech 


www.kontis.cz 


5 


0 


5 


Verkkosalkku, Verkko- 
opisto 


Finnish 




0 


1 


1 


Virtual-U 


Canadian 


www.vlei.com 


0 


1 


1 


Visit 






1 


0 


1 
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WebCT 


Canadian 


www.webct.com 


16 


4 


20 


Webleam Plus 






0 


1 


1 


West 






0 


1 


1 



Regional Preferences and Market Leaders 

In the countries that use English as their first language, the American LMSs seems to 
dominate: 

The overall impression is the domination of the scene by the major American-based LMSs, 
notably WebCT, Blackboard and TopClass. This is because of the use of English in the 
United Kingdom and Ireland. (Keegan 2002) 



WebCT has pushed hard to become the market leader with extensive promotion and 
presence at e-Leaming conferences. (Keegan 2002) 



In Australia, WebCT seems to be the most widespread LMS system and Blackboard seems 
to be the first runner up. A NCODE-FLA rhttp://ncode.mq.edu.au I LMS survey (NCODE- 
FLA, 2002) of 34 Australian institutions conducted by Sue McKnight shows 25 instances of 
WebCT, 12 instances of BlackBoard, and 7 instances of in-house developed LMS systems. 
This is supported by a briefing on leading learning platforms {The Observatory on 
Borderless Higher Mucation, 2002) which claims that Australia is the country with the 
highest penetration of BlackBoard and WebCT licenses in the world since 76 percent of the 
country’s 34 universities have such licenses. (Paulsen 2002 b) 



In the countries that not use English as their first language, the American LMS have many user 
institutions: 



The research indicates that ClassFronter, WebCT, FirstClass, and BlackBoard seem to be 
the most used LMS systems in the Nordic countries. (Paulsen 2002) 



The most used system is the Czech TUTOR2000 (currently by 5 institutions). 3 
interviewees have developed their own systems and the last 6 represent American 
commercially available LMSs (BlackBoard, Click21eam, GLN {Cisco}, Intraleam, Learning 
Space and WebCT). (Micincova 2002) 

But, the analyses show that the locally developed systems seem to have a strong position in the 
countries that not use English as their first language: 

ClassFronter is by far the market leader in Norwegian universities and colleges. Of those 
that offer online education, 65% used ClassFronter. (Runnesto and Ristesund 2002) 



Nordic institutions seem to prefer LMS-systems developed in the Nordic countries. Among 
the 25 different LMS systems that were identified in the analysis, 16 were of Nordic origin. 
All other systems were of American, Canadian, or Irish origin. (Paulsen 2002) 
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Institutions are slowly converting to the national LMS vendors, since the systems are in 
their mother tongue and want to be active in larger market and offer services to the general 
public. (Micincova 2002) 

Language is a main issue in Southern Europe and LMSs not translated to the countries' 
languages can be easily unsuccessful. (Dias, Dias, and Pimenta 2002) 

Vendor Specific Issues 

As the following statements show, the analyses indicate that there seems to be an overall 
satisfaction with the most used LMS systems: 

There seems to be general satisfaction with WebCT as a user-friendly, competent product. 
(Keegan 2002) 



Blackboard has given general satisfaction but is less widely marketed than WebCT. (Keegan 
2002) 



TopClass is praised for its student and records database. (Keegan 2002) 



There is a large extent of confidence among the users of ClassFronter with regard to the 
service offered by the contractor and the fast developing speed the program has had. The 
program has been updated with four new versions yearly, and many user requirements have 
been accommodated. The user reference group for ClassFronter is unique in Norway. This 
group is a decisive reason for why many institutions have selected ClassFronter. (Ruimesto 
and Ristesund 2002) 



Competitive Issues 



The findings that relate to the customers loyalty, user-fnendliness, cost effectiveness, 
integration, and opeimess could be of special interest to providers of LMS systems who want 
to compete in the future market: 



The institutions do not seem to be especially loyal to, or dependent on, one provider of 
LMS system. The majority of the institutions had changed system, planned to change 
system, or operated secondary systems. (Paulsen 2002) 



LMS systems could have reached a point were user-fiiendliness, cost effectiveness, and 
integration with other systems is more important than new features. (Paulsen 2002) 



The open source strategy may have an impact on the future LMS market. (Paulsen 2002) 



It should also be noted that many systems could be improved with regard to linguistic issues, 
assessment tools, pricing, content creation and management: 
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An overall evaluation allow us to observe that the commercially available platforms can be 
very practical to start with but they have problems with linguistic issues, as well as with 
assessment tools adequacy to target groups and pricing. (Dias, Dias, and Pimenta 2002) 



Most part of the systems researched seem to have problems with content creation and 
content management, students monitoring and assessment tools. Online administration and 
integration with other institution software and platforms were also a question. (Dias, Dias, 
and Pimenta 2002) 

Finally, it should be noted that adaptability and management facilities on the level above 
individual courses are requested: 

The unanimous data of LMS appearantly is "adaptability provided". (Fritsch and FoUmer 
2002) 



It is especially interesting to observe that several systems seem to lack facilities and services 
on the level above individual courses. (Paulsen 2002) 



Self-developed Systems 

Table 7 shows that the analyses revealed 35 different self-developed systems, and one may 
infer that there are remarkably many institutions that use self-developed LMS systems. 
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Table 7. Self-developed LMS systems included in the analyses 













Czech 




Czech-Swiss Institute 




Danish 




Danmarks Netskole 




French 




CNED 




German 




Netzentwurf 




German 




TU Chemnitz 




German 




Akademie 


ILIAS 


German 




Virtus 


Lemen-im-netz 


German 




Akademie 


Planetux 


German 




Virtuelle Universitat 


VC Prolog Tutor 


German 




Osnabrueck 


VU 


German 




LVU 


Wave learn 


German 




Darmstadt 




Greek 




Aristoteles University of 
Thessaloniki 


Aula virtual 


Italian 




Institute Foimazione Operatori 
Aziendali 


Experienze 


Italian 


www.esperienze.net 


Anonymous 


Greenteam 


Italian 


www.ereenteam.it/greenteam/educat 


Sinforml 


ion 


Proprietary 


Italian 




University of Trento 




Norwegian 




Hogskolen I Narvik 




Norwegian 




Hogskolen I Oslo 




Norwegian 




Hogskolen I Sor-Trondelag 




Norwegian 




Hogskolen I Stord/Haugesund 


Apollon 


Norwegian 




Handelshogskolen BI 


Ed-On-Web 


Norwegian 




Hogskolen I Vestfold 


Kurs.nlh.no 


Norwegian 




Norges Landbrukshogskole 


MvForum 


Norwegian 




Dronning Mauds Minne 


PedIT 


Norwegian 




Globalskolen 


SESAM 


Norwegian 




NKI Fjemundervisningen 


e-cursos 


Portuguese 


www.e-cursos.com 


Associagao Empresarial de Portugal 


Evolui 


Portuguese 




Prodigio 


Formare 


Portuguese 


www.formare.pt 


PT-Inovacao 


TWT Teaching 
Web Toolkit 


Portuguese 


www.mvtwt.net 


Universidade Catolica Portuguesa 






Slovak 




AINova 




Slovak 




LCDE 


ELIAS 


Spanish 


www.eleamine.uvigo.es 


University of Vigo 


SSVN2000 


Swedish 




Statens skolor fbr vuxna 





There may be many covert and vicarious reasons for choosing self-developed LMS-systems: 

There is a tradition that a bright computing center does not need to buy programs 
developed by others because the need to buy external programs would question the 
qualification of existing personel in such centers. (Fritsch and Fbllmer 2002) 



But the analyses indicate that institutions with self-developed LMS systems perceive the 
commercial systems as expensive and complex. The self-developed systems surpass linguistic 
problems and are regarded as supportive of special needs and target groups; 



Several institutions prefer self-developed systems. They perceive the commercial systems as 
expensive and complex and want to develop the systems to support their special needs. 
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They wanted cost effective systems with the ability to handle continuous enrolment and 
integration with student administrative systems and economy systems. (Paulsen 2002) 

Own developed systems are simpler and directly related to the target groups; they surpass 
the linguistic problems of the commercially available platforms and are constantly updated, 
being able to improve their features according to trainers, trainees and administration 
evolution. Besides the linguistic advantage national marketing strategies together with 
competitive pricing contribute to the great use of those own developed LMSs. (Dias, Dias, 
and Pimenta 2002) 



LMS functionality 



Course Development Tools 

The course content is usually developed with external tools before it is published in the LMS 
systems; 



LMS systems are usually not used for development of course content. A broad range of 
external tools is used to develop the content before it is published in the LMS system. 
(Paulsen 2002) 



To some interviewees LMS are mainly a form of support and sharing of information: Some 
institutions need to use external tools to the LMS and specialist support to course 
production, also suggesting that difficulties are based not in the platform but in the process 
of implementation. (Dias, Dias, and Pimenta 2002) 



In the majority of cases there is no course creation with or inside the LMS in fact 
independent of the origin of the LMS - self-developed or purchased. (Fritsch and Follmer 
2002) 



The analyses also indicate that there is a lack of available course content: 



The lack of available content is deplored. (Fritsch and Follmer 2002) 



Student and Tutor Support Tools 

There is a host of student and tutor support tools included in the LMS systems. However, the 
availability and quality of specific tools vary fi'om tool to tool. Here are a few statements that 
elucidate this: 

The student support tools available on the LMS are shown, generally, by the fora, chat 
mailing lists and email, having to bear in mind that not all the interviewed institutions use the 
so called services. Some have adopted a pedagogical model that discourages the use of the 
services of chat communication. Others include videoconference and collaborative 
technologies based on video streaming. (Dias, Dias, and Pimenta 2002) 

There is evidence from the data collected that not all LMS have facilities to monitor 
student’s performance, or when founded it is not enough for tutors’ task. Also student 
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administrative and background data is not directly accessible to tutors who need to make 
specific queries to the system administrator in order to get it. Some interviewees identify 
facilities of student tracking with the support of the system administrator. These 
observations lead us to conclude that in most part of the cases, teachers are not the ones 
who monitor the students. (Dias, Dias, and Pimenta 2002, 9) 

The concept of quizzes and multiple choice questioning, a feature of most American LMSs, 
is not considered adequate for European academic evaluation. (Keegan 2002) 

There is not one LMS using an integrated examination procedure. (Fritsch and Follmer 

2002) 



Administrative Systems 

The need for sophisticated administrative systems increases with the administrative workload: 



It seems a general trend that the administration facilities seem much more important for 
professional training institutions where (in general) courses are shorter, are "repeated", in 
several "editions", in short timelines, than in classical education institutions (universities), 
where the model adopted (longer, once a year) favours a more stable course / teacher / 
students association. (Dias, Dias, and Pimenta 2002) 



With the introduction of large-scale online education, the need for integration between LMS 
systems and other online education systems increases. The analyses revealed a general lack of 
such integration. It is however interesting to see that the Nordic universities have standardized 
on a few national student management systems and that interesting integration efforts are in 
progress: 

The Nordic universities have standardized on a few national student management systems. 

The systems are LADOK (Sweden), MSTAS (Norway), FS (Norway), STADS (Denmark), 

INNA (Iceland) and to some extent Oodi (Finland). (Paulsen 2002) 



There is a general lack of integration between the LMS and the student management 
systems. The analysis showed that some LMS has no possibility for integration, other has 
the possibility to import data fi'om the student management system, but only one system 
(SESAM, a self-developed system by NKI Distance Education) has full integration both 
ways. (Runnesto and Ristesund 2002) 



Data produced by the LMS systems are not yet generally integrated into the institutions' 
administrative databases. (Keegan 2002) 



LMS systems need to be integrated with a number of other systems in an organization that 
aims at providing efficient, large-scale, online education. (Paulsen 2002) 

The integration between the LMS systems and the student administrative systems seems to 
be relatively poor. (Paulsen 2002) 
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The integration between the LMS-systems and the economy systems seems to be very poor. 
(Paulsen 2002) 

Several of the interviewees are concerned about the opportunities and challenges regarding 
integration with the administrative system that records the student grades. (Paulsen 2002) 

A records or test-database is kept - if at all centrally - at a different place of the 
administration than the enrolment database. Because of the privacy laws of data protection 
it is not so easy to change these procedures. (Fritsch and Follmer 2002) 

Unlike in other countries we do find many projects, where the enrolment as a university 
student is the only prerequisite for participating in the LMS. It explicitly does not mean that 
the LMS is integrated into the normal university enrolment procedures : on the contrary, in 
most cases this will be completely separated. (Fritsch and Follmer 2002) 

Technology 

The analyses did not reveal important technical problems with the LMS systems. There were 
however many interviewees who expressed a wish for higher bandwidth to be able to provide 
more multimedia content and services: 

Concerning the technological aspects, in general, the access to the LMS is through Internet, 
and the speed of the system depends on the limits of bandwidth available for each user. 
Users are accepting this situation as satisfactory, except two interviewees; one uses video 
contents on their courses, and the other has a large experience in corporations with their 
own network. (Dias, Dias, and Pimenta 2002) 



Neither the limitation of student numbers nor the limits of number of courses provided are 
real problems for the systems interviewed. (Fritsch and Follmer 2002) 

As expected the speed provided by the systems is much higher than the velocity of learners 
systems at large, fritsch and Follmer 2002) 

Economic Issues 

Cost effectiveness becomes more important as the institutions become large-scale providers of 
online education. The interviewees have, however, vague knowledge about the system's 
maintenance and operation costs. The cost and pricing structure for the commercial systems 
vary from system to system. This could make it difficult to compare real costs. Some 
interviewees were considered about high and increasing prices for the commercial LMS 
systems. The following findings relate to these economic issues: 

Recent price increases, often quite considerable, are a feature of the market with prices in 
the range € 20.000 to € 50.000 being quoted. (Keegan 2002) 

The costs and pricing structure for the commercial systems vary from system to system. 

This could make it difficult to compare the real costs. (Paulsen 2002) 
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Many interviewees mentioned that economic aspects are hard to identify. However, those 
who gave objective values, converged for a staff team of around 2~3 people full-time 
(technicians plus help desk), plus a variable size support team depending on the number of 
users who need help. Teachers / tutors are also an additional cost, they usually require some 
initial training in using the LMS. (Dias, Dias, and Pimenta 2002) 

The institutions are facing significant financial problems, are afi'aid of announced charge 
increases of the LMS. The system development stagnates because of this lack. Faster 
implementation of the eEurope+ targets might have been reached if this implementation 
would not be largely coming from national budgets in the candidate countries. (Micincova 
2002 ) 



Installing a complete system mostly meant to buy a new machine and database software, 
which easily sums up to some €100.000 Euro. But the answers of the majority of projects 
■will hide these cost behind the statement that it is self-developped, open source, or not 
available (n.a.). (Fritsch and FoUmer 2002) 



Overall Evaluation 

LMS systems are not able to handle all the functions the institutions want, and they could be 
improved in many ways. But most systems encountered in the analyses seem to be good 
enough for handling online education successfully: 



At present there is a tendency to organize and structure the e-Leaming offer using a type of 
software somewhere classified as Learning Management System. Those systems are 
dedicated to some issues of the learning process, but in almost all the cases (commercially 
available systems or self developed) the systems are not able to perform all the activities the 
institutions need. Administration aspects, integration with existing software and content 
management are some of the issues not well treated by most part of the LMS studied. (Dias, 
Dias, and Pimenta 2002) 



There were cases in the study, where a kind of facility of a system was not available (e.g. 
synchronous communication). It must be considered that the institutions are choosing their 
system and its functionalities according to their future planned activities and requirements. 
That means that in spite of the fact that the system seems to have a shortage it is actually 
not the case, because the system in that specific form is the most suitable and fully 
satisfactory for the institution. (Micincova 2002) 



Knowledge, Policy, and Strategy 

The analyses indicates that there is a need for enhanced focus on LMS knowledge, policy, and 
strategy in Southern Europe; 

Another interesting observation is the lack of a common understanding concerning terms 
and functionalities of the LMS systems. (Dias, Dias, and Pimenta 2002) 



Another important issue is the universities e-Leaming managers concern with the university 
policy and strategies for this field. Apparently Southern European universities are not 
dedicating enough importance and attention to this subject. (Dias, Dias, and Pimenta, 2002) 
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Another important issue is the universities e-Leaming managers concern with the university 
policy and strategies for this field. Apparently Southern European universities are not 
dedicating enough importance and attention to this subject. (Dias, Dias, and Pimenta, 2002) 

The analyses further indicate that the introduction of LMS systems could be a source for 
conflict between administration and academia. 

There seems to have emerged a new dichotomy that plays a major role in the development 
of web education systems. The dichotomy between academia and administration. (Fritsch 
and Fbllmer 2002) 



The task of such a computing center indeed will necessarily produce a conflict between the 
university's administration which hoped for increase of effciciency and decrease of 
performance cost and the department which always has to look for proofs of their 
respectability tested in - at least- the "own" market. (Fritsch and Fbllmer 2002) 



E-learning Standards 



The institutions in North Western Europe are sensitive to the e-leaming standards and they are 
considered almost as a norm. The Nordic interviewees are aware of the standards, and several 
claim to follow them. But few state that the standards are important to their institution, and e- 
leaming standards do not seem to have had much impact on online education in the Nordic 
countries. The German analysis states that standardization will play an important role in the 
future. In Southern Europe it also seems to be a considerable ambivalence with regard to e- 
leaming standards. 



The interviewees are aware of the e-leaming standards, and several claim that their systems 
follow the standards. But few state that the standards are important to their institution, and 
e-leaming standards do not seem to have had much impact on online education in the 
Nordic countries. (Paulsen 2002) 



All institutions are sensitive to SCORM and IMS standards and dissemination of these 
standards has led to them being considered almost as a norm. (Keegan 2002) 



Summarized one can say that questions of standardization and optimized possibilities of 
synchronous communication with different tools will play an important role in the future. 
(Fritsch and Fbllmer 2002) 

In relation to Standards, interviewees stressed the absence of both 'de facto' and 'proposed', 
technical standards. Curiously, (only) one interviewee mentioned his/her belief that 
standardization will have a positive impact in internationalization of eLearning businesses; 
and, on the other hand, one other person stated the complementary idea "Since our courses 
are country specific, standards are not yet relevant". The existence of standards is welcome, 
both for marketing reasons (as indicated before), but also for cost reduction ('rationalization 
of resources') or LMS migration. (Dias, Dias, and Pimenta, 2002) 

BEST COPY AVAILABLE 



VRIC 



Morten F. Paulsen & Desmond Keegm: European Experiences with Learning Management Systems 



Ten Important Findings from a European Perspective 

Ten important findings result from the research reported here. These findings will be of value 
to: 



• Representatives of the European Commission and international planning agencies 

• National government agencies planning for e-Leaming provision in their systems 

• Developers and vendors of Learning Management Systems 

• Universities and training institutions considering entering the e-Leaming market or 
considering upgrading their existing systems. 

1. There are significant regional differences within Europe with regard to how far the 
institutions have come in their use of LMS systems. The differences seem to follow the 
regional statistics for Internet users, which means that the Southern Europe, the Czech 
Republic and Slovakia seems to be less mature with regard to use of LMS systems than the 
other regions. 



2. The analyses indicate that there is a clear trend that institutions offer more online courses 
today than they did three years ago. One may say that the trend is to go from small-scale to 
large-scale online education. If one characterizes institutions that offer at least 50 online 
courses as large-scale providers of online education, 30 of the 89 institutions (34%) we have 
data from could be characterized as large-scale providers. The analyses indicate that the trend 
towards large-scale online education has come further in the Nordic countries (60%) than in 
the other regions. 



3. The analyses indicates that the BlackBoard, ClassFronter, FirstClass, Lotus Learning Space, 
LUVIT, TopClass, Tutor2000, and WebCT are among the most used commercial LMS 
systems in Europe. 



4. The analyses found four European LMS systems that seem to be significant competitors on 
the European market. TopClass originated as a European Commission project at the University 
College Dublin, in Ireland, before becoming an Irish campus company and then migrating to 
the United States. ClassFronter is a Norwegian developed system that has a very dominant 
position in Norwegian universities and colleges. The system is available in a number of 
languages and sold to institutions in several countries. LUVIT originated at the University of 
Lund in Sweden, before it became a Swedish commercial company with reasonable success in 
Scandinavia and some other countries. Tutor2000 seems to be a successful LMS provider in 
the Czech Republic. 



5. A striking conclusion of this study is that the generally accepted position that the market is 
dominated by the American LMSs, is not the norm throughout Europe. In the countries that 
not use English as their first language, locally developed LMS systems have successfully 
repelled the American products. A remarkable large number of the LMS systems used in 
Europe are commercial systems developed locally or self-developed systems at the institutions. 
However, very few of these systems seem to have more than a few user institutions. 

6. There are remarkably many institutions that use self-developed LMS systems, and there may 
be many covert and vicarious reasons for choosing self-developed LMS-systems. But the 
analyses indicate that these institutions perceive the commercial systems as expensive and 
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complex. The self-developed systems surpass linguistic problems and are regarded as 
supportive of special needs and target groups. 

7. With the introduction of large-scale online education, the need for integration between LMS 
systems and student management systems increases. The analyses revealed a general lack of 
such integration. It is however interesting to see that the Nordic universities have standardized 
on a few national student management systems and that interesting integration efforts are in 
progress. 

8. Cost effectiveness becomes more important as the institutions become large-scale providers 
of online education. The interviewees have, however, vague knowledge about the system's 
maintenance and operation costs. The cost and pricing structure for the commercial systems 
vary from system to system. This could make it difficult to compare real costs. Some 
interviewees were considered about high and increasing prices for the commercial LMS 
systems. 



9. The analyses indicate that there is a need for enhanced focus on LMS knowledge, policy, 
and strategy in Southern Europe. In particular universities e-leaming managers are concerned 
with the university policy in this field. Apparently they mean that Southern European 
universities are not dedicating enough importance and attention to this subject. The analyses 
further indicate that the introduction of LMS systems could be a source for conflict between 
administration and academia. 



10. The institutions in North Western Europe are sensitive to the e-leaming standards and they 
are considered almost as a norm. The Nordic interviewees are aware of the standards, and 
several claim to follow them. But few state that the standards are important to their institution, 
and e-leaming standards do not seem to have had much impact on online education in the 
Nordic countries. The German analysis states that standardization will play an important role in 
the future. In Southern Europe it seems to be a considerable ambivalence with regard to e- 
leaming standards. 
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Online Education Systems: Definition of Terms 

By Morten Flate Paulsen 
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April 2002 



This paper is written in order to establish a common framework of terms for the Web 
Education Systems Project (Web-edu) which is supported by the European Leonardo da Vinci 
program. The project’s web-pages are available at 

www.nettskolen.com^inenglish/webedu.html . 



The following main terms are presented and explained in a logical sequence: 



Online education 
Online education systems 
Content Creation Tools (CCT) 
Learning Management System (LMS) 
Student Management System (SMS) 
Accounting System (AS) 



Online Education 

There are many terms for online education. Some of them are: virtual education, Internet-based 
education, web-based education, and education via computer-mediated communication. The 
project uses a definition of online education that is based on Desmond Keegan's definition of 
distance education. Hence, online education is characterized by: 



the separation of teachers and learners which distinguishes it from face-to-face 
education 

the influence of an educational organization which distinguishes it from self-study and 
private tutoring 

the use of a computer network to present or distribute some educational content 
the provision of two-way communication via a computer network so that students may 
benefit from communication with each other, teachers, and staff 



Online Education Systems 

Online education systems are defined as all systems that support online education. In the 
following, this paper discusses two different models for online education systems developed by 
the web-edu project. The models are: 

- The Jigsaw model for online education systems 
The Hub model for online education systems 



The Jigsaw Model for Online Education Systems 

The jigsaw model is a simplistic model used in the web-edu project. It t includes the four main 
categories of online education systems that are listed below and presented in Figure 1 . 



Content Creation Tools (CCT) 
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Learning Management System (LMS) 
Student Management System (SMS) 
Accounting System (AS) 



It is called the jigsaw model to indicate that these systems should fit together to exchange data 
more or less seamlessly. 

Figure 1. The jigsaw model for online education systems 
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Content Creation Tools (CCT) 

Content creation tools are the tools that course designers and teachers use to create the 
content in online education courses. The content creation tools are used to develop learning 
material. There are many types of content such as for example plain text, slides, graphics, 
pictures, animations, audio, video etc. Typical examples of theses systems are DreamWeaver, 
Frontpage, Word, PowerPoint, and Director. These are generic tools with few features 
developed specially for online education. 



In addition to the much-used generic CCT tools, there are a number of CCT tools that are 
specially made for development of educational content. These CCT tools are termed authoring 
tools. 



Authoring Tools 

Authoring tools could be regarded as a subset of content creation tools. Brandon Hall (2001) 
defines an authoring tool as ”a software application, used by non-programmers, that utilizes a 
metaphor (book, or flow chart) to create on-line courses”. One may say that authoring tools 
are content creation tools that are especially developed for creation of educational content. 



Learning Management System (LMS) 



Learning Management System is a broad term that is used for a wide range of systems that 
organize and provide access to online learning services for students, teachers, and 
administrators. These services usually include access control, provision of learning content, 
communication tools, and organizations of user groups. 
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Some examples ofLMS systems are WebCT, Blackboard,... 

Brandon Hall (2001) presents the following definition: 

A Learning Management System (IMS) is software that automates the administration of 
training events. All Learning Management Systems manage the log-in of registered users, 
manage course catalogs, record data from learners, and provide reports to management. 

There used to be a distinction between Learning Management Systems and more powerful 
Integrated Learning Management Systems. That distinction has now disappeared. The term 
Learning Management System is now used to describe a wide range of applications that 
track student training and may or may not include functions such as: 

Authoring 

Classroom management 

Competency management 

Knowledge management 

Certification or compliance training 

Personalization 

Mentoring 

Chat 

Discussion boards 

Virtual Learning Environment (VLE) 

Virtual learning environment is a term that to some extent is used instead ofLMS. The two 
terms have more or less the same meaning, but one may argue that VLE focus less on the 
features related to the management of learning. Bandon Hall (2001) defines learning 
environment this way: 

A Learning Environment is software designed as an all-in-one solution that can facilitate 
online learning for an organization. It includes the functions of a learning management 
system for those courses within the learning environment, but it may not be able to track 
online courses that were not created within this particular learning environment. 

A learning environment is characterized by an interface that allows students to register and 
take courses, staying within that environment for the duration of the course. The program 
will usually include some self-instructional portions, along with an academic model of a 
multi-week course. This model is often facilitated by an instructor, where a group can 
proceed on a week-to-week basis with seminar assignments. Most learning environments 
also include an authoring capability for creation of additional courses for the instructor. 

Student Management System (SMS) 

The student administration system is the core system in an educational institution. It is used for 
management of the most pivotal information about entities such as students, faculty, courses, 
applications, admissions, payment, exams, and grades. An effective SMS system is crucial for 
all educational institutions. 
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Enterprise Resource Planning (£RP) systems or Human Resource Information Systems 
(HRIS) 

Companies and corporations have comparable systems that hold important information about 
their employees. These could be termed Enterprise Resource Planning (ERP) systems or 
Human Resource Information Systems (HRIS). These systems will provide some of the same 
functionalities as the student m^agement systems. 

Some examples of SMS systems are PeopleSofl, Banner, MSTAS, FS, andLadoc. 



Brandon Hall (2001) provides the following descriptions of ERP and HRIS systems: 



Enterprise Resource Planning (ERP) is an industry term for large, often multi-module 
software applications that manage many facets of a company's operations including product 
planning, parts purchasing, maintaining inventories, interacting with suppliers, providing 
customer service, tracking orders, and managing resources and financials. SAP, PeopleSofl, 
and J.D. Edwards are some well-known ERP providers. 



Human Resource Information Systems (HRIS) are similar to ERP applications but are 
aimed specifically at the management of a company's human resources. 



Accounting System (AS) 

The accounting system is used for recording the economic transactions between the institution 
and its customers and suppliers. In an online education setting, the most important customers 
and suppliers are the students and the teachers. 

The data from the accounting systems should be used to deny system access to students who 
do not pay their tuition fees. Some institutions already accept online enrollment, online 
payment, and online student credit account information. Other institutions provide online tutors 
with their updated salary account information. This functionality requires some integration 
between the systems. 



The Hub Model for Online Education Systems 

The hub model is more complex than the jigsaw mode. It is included to show that online 
education systems are becoming more and more complex. This is partly due to the institutions’ 
need to rationalize the operation to handle the growing number of online students and courses, 
and partly due to the fact that the users are increasingly expecting more sophisticated services. 



The model is termed the Hub Model to indicate that the Student Management System is the 
central, most important system for large-scale online education. For historical, legal, and 
financial reasons, the SMS system is the most important system for an educational institution. 
Hence, all other systems that could offer online education services should rely on the SMS 
system as the master system with which they exchange data. 
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Figure 2. The Hub model for online education systems 
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Related Terms 

Brandon Hall (2001) explains a number of related terms: 

Content Management Systems (CMS) are used to store and subsequently find and retrieve 
large amounts of data. Content Management Systems work by indexing text, audio clips, 
images, etc., within a database. In addition, CMS often provide version control and check 
in/check out capabilities. Using robust built-in search capabilities, users can quickly find a 
piece of content fi-om within a database by typing in keywords, the date the element was 
created, the name of the author, or other search criteria. 



Content Management Systems are often used to create information portals for organizations 
and can serve as the foundation for the practice of knowledge management. They can also 
be used to organize documents and media assets. For example, a newspaper agency may use 
a content management system to provide an archive of every story ever written for the 
paper. Likewise, they might use the CMS to provide an extensive library of photographs 
that are reusable for future stories. 



A learning content management system is an environment where developers can create, 
store, reuse, manage and deliver learning content fi'om a central object repository, usually a 
database. LCMS generally work with content that is based on a learning object model. 
These systems usually have good search capabilities, allowing developers to find quickly the 
text or media needed to build training content. 



Learning Content Management Systems often strive to achieve a separation of content, 
which is often tagged in XML, from presentation. This allows many LCMS to publish to a 
wide range of formats, platforms, or devices such as print, Web, and even Wireless 
Information Devices (WID) such as Palm and windows CE handhelds, all from the same 
source material. 
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Learning Management Systems (LMS) used in Southern 
Europe 

By Ana Dias, Paulo Dias and Pedro Pimenta 

Abstract 

This part of the study refers to Southern Europe. Research and interviews were 
applied to a number of institutions, experienced in e-Leaming and using Learning 
Management Systems (LMS) to support their e-courses provisions and e- 
Leaming structures. 

A total of 21 interviews were carried out all over the concerned countries - 
Portugal, Spain, Italy, France and Greece. 

A Southern Europe overview puts in evidence the regions figures in terms of 
Internet users. In fact 17,5% of Southern European population uses Internet, much 
less then the 50% of Internet users in the Nordic Countries, the 33% in North- 
western Europe, the 30% in Germany, 10% in Czech Republic (similar to Portugal 
and Greece). 

Interviews include 1 1 self developed LMSs and 5 commercially available LMSs. 
From the commercially available LMSs in use, 4 Institutions use Web - CT (3 are 
Universities), 2 other Institutions use Blackboard, 2 other use Learning Space and 
one institution uses Docent and another institution uses Intraleam. 

There are a total of 561 online courses that are using the 16 LMS systems. 
Important is to note that 4 institutions have from 1 to 3 courses, 7 institutions have 
from 4 to 15 courses and only 10 have from 16 to 85 courses. These mean that 
50% of the researched institutions have less then 15 courses online. 

There are a total of 470 online tutors and a total of 41296 only students. 

At present there is a tendency to organise and structure the e-Leaming offer using 
a type of software somewhere classified has Learning Management System. Those 
systems are dedicated to some issues of the learning process, but in almost all the 
cases (commercially available systems or self developed) the systems are not able 
to perform all the activities the institutions need. Administration aspects, 
integration with existing software and content management are some of the issues 
not well treated by most part of the LMS studied. 

Language is a main issue in Southern Europe and LMSs not translated to 
Countries languages can be easily unsuccessful. 

The research also led us to observe that most part of the e-Leaming managers 
assume a position of experimentation and initiation on the e-Leaming process 
(50% of the researched institutions have less then 15 courses online). 

An overall evaluation allows us to observe that the commercially available 
platforms can be very practical to start with but they have problems with 
linguistic issues, as well as with assessment tools adequacy to target groups and 
pricing. 

Own developed systems are simpler and directly related to the target groups; they 
surpass the linguistic problems of the commercially available platforms and are 
constantly updated, being able to improve their features according to trainers, 
trainees and administration evolution. Besides the linguistic advantage national 
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marketing strategies together with competitive pricing contribute to the great use 
of those own developed LMSs. 

Another important issue is the Universities e-Leaming managers concern with the 
University policy and strategies for this field. Apparently Southern European 
Universities are note dedicating enough importance and attention to this subject. 



1. Methodology 

This part of the study refers to Southern Europe. Research and interviews were 
applied to a number of institutions, experienced in e-Leaming and using Learning 
Management Systems (LMS) to support their e-courses provisions and e- 
Leaming structures. 

This analysis is based on in-depth, qualitative interviews applied to leading e- 
Leaming Managers from institutions located in Southern Europe. A total of 21 
interviews were carried out all over the concerned countries - Portugal, Spain, 
Italy, France and Greece. The interviewees were selected first by the 
demonstrated experience in the e-leaming field and then according to their 
availability to answer to the interviews. The interviews were carried out via e- 
mail, phone or face-to-face. The researchers involved in the interviews used also 
the institutions web pages and included information collected from there. Some 
of the interviews were made in Portuguese, Spanish and Italian and then 
translated into English. Other interviews were done directly in English (France 
and Greece). 

The interview guide was developed by the international research team according 
to criteria of functionality and utility of the research. 

The interview guide was focusing is attention on the following topics; 

Institutions and LMSs; 

Course Development Tools; 

- Tutors support tools; 

- Administrative systems; 

- Technology; 

- Price; 

- Overall evaluation; 

- Future features to include in existing LMS systems 



2. Southern Europe Overview 

Southern Europe can be characterised as a region Avith different countries, 
communicating in different languages in a relatively small geographical area. 
Portugal, Spain, France, Italy and Greece occupy an area of 1,577,373 Square 
Kilometres, with a population of 177,6 Million People, speaking 5 completely 
different official Languages, independently of other less spoken languages in 
those countries. 

According to data available from Eurostat and presented in table 1, there are 31 
Million of Internet users in Southern Europe, representing 17,5% of the 
population. 
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Table 1. Countiy, Official Language, Inhabitants and Internet penetration 



Country 


Language 


Area (sq 

km) ' 


Inhabitants 

(millions)^ 


Internet hosts 

(per 100 

inhabitant)^ 


Internet users 

(per 100 

inhabitant) 


Portugal 


Portuguese 


92 391 


10.2 


1.2 


10.0 


Spain 


Castilian 


504 782 


39.5 


1.4 


13.9 


France 


French 


547 030 


59.5 


1.7 


16.9 


Italy 


Italian 


301 230 


57.8 


2.7 


23.3 


Greece 


Greek 


131 940 


10.6 


1.0 


9.5 


Total 




1,577,373 


177,6 







The number of Internet users, compared with other European Regions and with 
USA is very low in Southern Europe. Nordic Countries have 50% of the 
population using the Internet and the USA 56,5%. 

Table 2 shows data referring to some European Regions and USA. 

Germany with has nearly 30% of the population using Internet, Czech Republic 
can be compared to Portugal or Greece, having a population of around 10 
Million People and having around 10% of the population using Internet. 

North Western Europe occupy an area of 315 100 Square Kilometres, with a 
population of 63,6 Million People, all are speaking English as main language for 
communication, independently of other less spoken languages in those countries. 
There are 21 Million Internet users in North Western Europe, representing 33% 
of the population. 

Nordic Europe occupy an area of 1 154 308 Square Kilometres, with a 
population of 23,9 Million people, speaking 4 different languages (Nordic 
Languages are all different but very similar in their route. In average people from 
different Nordic countries are able to understand each other). According to data 
available and presented in table 2, there are 12 Million Internet users in Nordic 
Europe, representing 50% of the population. 



Table 2 Country? Official Language, Inhabitants and Internet penetration 



Country 


Language 


Area (sq km) ^ 


Inhabitants 

(millions)^ 


b/B BB^B CB B 

Internet hosts 
(per 100 inh)^ 


BVXX 

Internet users 
(per 100 inh) 


Portugal 


Portuguese 


92 391 


10.2 


1.2 


10.0 


Spain 


Castillian 


504 782 


39.5 


1.4 


13.9 


France 


French 


547 030 


59.5 


1.7 


16.9 


Italy 


Italian 


301 230 


57.8 


2.7 


23.3 


Greece 


Greek 


131 940 


10.6 


1.0 


9.5 














Germany 


Deutsch 


357 021 


82.2 


2.3 


29.6 














Czech Republic’ 


Czech 


78 866 


10.3 


1.6 


9.7 














Ireland 


Irish 


70 280 


3.8 


2.3 


27.5 


United Kingdom 


English 


244 820 


59.8 


3.5 


33.5 














Norway 


Norwegian 


324 220 ' 


4.5 


11.2 


52.7 



' Data from CIA World Factbook (http://www.odci.gov/cia/publications/factbook/. 6 of July 2002) 

^ Data from "People in Europe", Eurostat, 2002, referring to January l”, 2001. 

^ Data from "Enterprises and their activities", Eurostat, 2002, referring to 2000. 

^ Data from CIA World Factbook (http://www.odci.gov/cia/publications/factbook/. 6 de Julho de 
2002) 

^ Data from "People in Europe", Eurostat, 2002, referring to January l“, 2001. 

® Data from "Enterprises and their activities", Eurostat, 2002, referring to 2000. BEST COPY AVAILABL 
’ Data from "The candidate countries", Eurostat, 2002, referring to 2000. 
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Sweden 


Swedish 


449 964 


8.9 


7.0 


56.4 


Finland 


Finish 


337 030 


5.2 


13.6 


44.5 


Denmark 


Danish 


43 094 


5.3 


13 


43 














USA 


English 


9 629 091 


278.1 


29.3 


55.8 



From this analysis we would like to emphasise the differences between the 
Northern Europe Internet uses and the Southern Europe ones. In fact we range 
from a 50% of Internet users in the Nordic Countries to 33% in North-western 
Europe, 30% in Germany, 10% in Czech Republic and 17,5% in Southern 
European countries. These figures will reflect on the Countries and Regions level 
of development in the e-Leaming field. 

Table 3 represents a summary of the important relations between area, population 
and Internet users in different European Regions. 



Table 3 European Region, Area, Inhabitant 


ts and Internet users 


Region 


Area (sq km) 


Inhabitants 

(millions) 


Internet users 
(% Inhabitants) 


Southern Europe 


1 577 373 


177,6 


17,5% 


Germany 


357 021 


82,2 


29,6% 


Czech Republic 


78 866 


10,3 


9,7% 


North Western Europe 


315 100 


63,6 


33% 


Nordic Europe 


1 154 308 


23,9 


50% 



This evidence about North - South disparities is recognise by the European 
Commission in the article ’’The digital divide; disparities between member states”, 
downloaded from the europa web site*: 

” The eEurope benchmarking report put in evidence the high disparities between 
Member States in IT progress. The Report shows considerable differences 
between countries in almost every item analysed. Not amazingly, there is a group 
of e-advanced countries ranking high in almost all indicators, while the less IT 
advanced States rank poorly in many aspects. A conclusion of the Report is that 
"many Member States are too far behind leading EU Member States in Internet 
penetration and use. More efforts are needed to close the gap." 

Differences in Internet and broadband take-up reflect an EU North-South digital 
divide 

Other indicators show a similar geographic pattern. There are more than 20 PCs 
connected to the Internet per 100 pupils in the schools of Denmark and 
Luxemburg, and more than 10 in Finland and Sweden, while 8 countries rank 
below 5 PCs per 100 pupils, with Spain, Germany, Italy, Greece and Portugal in 
the last positions. The more advanced countries show their pre-eminence in 
several indicators, so not surprisingly Denmark, Sweden and Finland, for instance 
are the 3 leading countries in the highest percentage of workers having computer 
training. 

Several reasons could be advanced to explain this gap, including cultural aspects. 
The document " Benchmarking National and Regional E-Business Policies ", from 
the Enterprise Directorate finds that in countries like Spain, Italy and Greece "a 
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lack of e-business awareness is still a major hurdle to its widespread adoption". 
So probably attitude has a role to play in order to explain such EU disparities. . . . 
Probably the European North-South North-South digital gap should be regarded 
as a temporary effect due to different growth rates and unequal starting points, so 
it is expected to be reduced in the near future. At least the general trend shows 
strong progress in economic and social cohesion in the EU. The First Progress 
Report on economic and social cohesion, presented on 30 January 2002 by the 
European Regional Policy Commissioner Michel Bamier, confirms that "the least 
prosperous countries (Spain, Greece and Portugal) are indeed catching up: the 
extent to which they originally lagged behind has shrunk by nearly one third in the 
last 10 years, with per capita GDP rising fi-om 68% to 79% of the Community 
average". But lastly, although the progress done, the Report shows that "the gaps 
between countries and regions in Europe in terms of population density, levels of 
education and access to new technologies are still too wide." 

Bellow we describe some Southern European characteristics and cultural factors 
which can began to give us a picture of the context: 

Southern Europe is an area highly populated with 5 countries and 
speaking 5 different languages (Portuguese, Spanish, French, Italian and 
Greek); 

In all Southern European Countries the urban areas are much more 
populated then rural areas; 

5 different languages are spoken, besides less spoken languages in those 
countries 

- Due to historical reasons, some of the Southern European Countries, 
have developed particular and privileged relations with a set of other 
countries, which share the same language (Portuguese speaking 
Countries, Spanish speaking Countries, Francophone Countries); 

- In Southern Europe Countries, Distance Education in the classical sense 
(correspondence courses and TV based teaching) was not very much 
developed mainly due to peoples physical closeness; 

- Professionals are used to get training fi-om their local training centres, or 
travel and get training in other training centres outside their hometown; 

- In general training is provided in their own country and using their own 
country language; 

- We can say that it is rare to find a professional applying for training in 
other countries, not only because of the language difference, but also due 
to the distance and to the costs involved. 

- In the case of large companies or multinational companies, professional 
are induced to learn other languages, and it is common for them to learn 
or apply for learning in other countries; 

For the propose of providing a better understanding of Southern European 
specificities we provide a short overview of the e-leaming field in the countries 
involved is this study. 

2.1 Portugal 

Portuguese Distance Education is a very recent phenomenon. The Portuguese 
Universidade Aberta® was created in 1988 and is the Portuguese Public 
University devoted to Distance Education. 



' Open University of Portugal 
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More recently, with the emergence of new information and communication 
technologies, other universities, oriented towards face-to-face studies, have 
developed a set of activities in the online education field, especially using Internet 
tools. 

Three years ago Ana Dias (Cisaer, 2000) described the Portuguese scene in this 
field: ’’Online education in Portugal is a very unstructured domain, at both public 
and private level. 

There are no national bodies directly concerned with this issue and the 
universities that should be responsible for that development are starting their 
work in the field. ! 

This means that the offer of distance learning courses using the web is not 
structured neither in the public institutions, universities or training centres. In 
fact, the Cisaer (1999) Survey shows that in Portugal there is no institution, 
which offer courses online in a systematic way. In almost all analysed cases, the 
web courses provided are developed in the context of pilot projects, financed by 
the European Union and with a limited life span. 

INOFOR - the Institute for Innovation in Training is the single national body that 
expresses its concern with the emerging issues related to distance learning 
provision via the web. In the last year there was a growing number of institutions 
involved in e-leaming activities. In particular Training Institutions and 
Universities, which are migrating to e-Leaming provisions in a large speed and 
are organising their Education and Training offer also in an online basis.” 

Today it is not yet clear how will the training and education markets evolve in 
this field but institutions are purchasing commercially available systems or 
develop their own systems and are experiencing their use with different target 
groups. We can refer to an impressive evolution of the e-Leaming market into an 
organised and structured direction in the last two years. 

The most part of the Universities are using Web Ct as platform for distributing 
their learning, either internally to their face-to-face students either to the 
Continuous Training market. The penetration of Web CT in the Universities is 
impressive. 

The private companies are selling either Portuguese solutions or solutions 
adapted to Portuguese language and market. This aggressive policy is leading 
some bigger companies, to leading positions in terms of penetration of their 
LMSs in the training market. 



2.2 Spain 

Three years ago, the Cisaer study reported background information about 
Distance Learning and the online education field in Spain. In this worldwide 
study Ana Dias (2000) wrote: 

’’Distance Education in Spain, was initiated in the seventies. UNED*°, the Spanish 
Open University, was created in 1972 aiming to improve the cultural progress 
and to socially develop the country in a perspective of equal opportunities. 
Besides UNED, another institution was traditionally responsible for the large 
development of distance education in Spain: CEPADE (the Post-graduate Centre 
in enterprise management studies, directly dependent on the Madrid Polytechnic 
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University General Foundation). CEPADE was one of the first e-leaming 
structures in Spain, which was using First Class in a large scale. 

In the nineties, a new and very innovative university was bom and revolutionised 
the distance education scene: the Universitat Oberta de Catalonya (UOC). 
Created by the Regional Government of Catalonya, this university had a new 
vision, to put information and communication technologies at the service of 
students and teachers: Having more then 10 000 students all over the world, 
UOC developed its own LMS system called ’’Virtual Campus”. The participants 
are able to communicate amongst themselves, with the tutor and with the 
electronic environment, having access to fomms and chats. The students also 
have access to a Virtual Library and to the bibliographical UOC database. 

Online education evolved rapidly in Spain and there are a large amount of face- 
to-face universities, training centres and private companies adopting this t 3 T>e of 
learning.” 



Today Spain has a wide and spread e-leaming market. A simple search using a 
search engine like Google or Yahoo, searching for ”e-learning in Spain” can 
provide us with a wide range of entrances with web sites selling e-leaming 
solutions and institutions using e-leaming as a mean for distributing e-courses. 



2.3 France 

Only two years ago, anyone asking about e-leaming in France would have gotten 
a very confused stare as an answer. Ana Dias (2000) wrote about the French 
online education field: 

”In France there are different structures organising and promoting online 
education, on the basis of pilot projects, either European or national. The Ministry 
of Education, Research and Technology" has set up a National Educational 
Network that allows individual users to access pedagogical multimedia material 
available on the web. . 

At the educasource web site (www.educasource.education.frI and at the 
didasource web site (www.cnpd.ff/didasourceA ). it is possible to find an 
information system with pedagogical material and online documents. 

The French "country regions"" have also developed different strategies to 
promote online education, and there are other representative boards with web 
presence. Among others we can refer: 

• GEMME (Ebgher Education Group for Mediated Teaching"), which groups 
higher education institutions and puts documents dedicated to open and 
distance learning available online; 

• the Paris Chamber of Commerce PREAU - a lab dedicated to the new 
educational technologies; 

• THOT the Internet service-news, distance learning to French speaking 
countries. 

In the regional contexts we can also highlight the open and distance learning 
network created in the Rhone-Alpes area. This regional network regroups 
AFPA" (3 sites), CNAM" (4 sites), CNED (2sites), GRETA" (9 sites). It is a 
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network based on the voluntarism of its members, where the service is driven 
from the local community needs. 

CNED, the National Distance Education Centre*’ has recently created its 
electronic campus**, offering communication tools to students as well as the 
possibility to work on exercises and to be in permanent contact with the tutor. 

In France, like in the rest of Europe, European pilot projects are responsible for a 
large part of the market movements. 

The online education field is evolving rapidly in France and there is a large 
amount of face-to-face universities, training centres and private companies 
adopting this type of learning.” 



Today French Universities and Training Organisations are using e-leaming 
strategies. French Companies and corporations are offering online training to 
employees. 

Some Universities are gather in the Interuniversity Federation for Distance 
Learning (FIED), other Universities have signed an agreement with CNED, and 
others have signed an agreement with the CNAM. Furthermore there are 
Universities’ Continuing Education Departments or other Higher Education 
Institutions, which have formed alliances either amongst themselves or with 
private-sector partners to create digital campuses. 

French Ministries of Education and Research promoted two key initiatives in the 
ODL field, the Form@,sup (htt p://www.formasup.education.fr/index.php') . an 
information server for Higher Education ODL programmes and the call for 
projects to create French Digital Campus 
(http://www.educnet.education.fr/superieur/campus.htm') . 

It also has numerous teaching resources, which can be accessed directly online, as 
well as links to other resource sites such as EducaSup 
(http://www.educasuD.education.frA) and Educasource 
(http ://www. educasource. education. frA) . 

A call for projects to create Digital Campuses was launched in 2000. These 
projects are intended to provide a selection of post-secondary ODL courses, 
which use new information technologies for clearly identified fields and target 
groups. This call for projects led to the selection of 49 projects in 2000. Another 
66 were chosen in 2001 

(http://www.educnet.education.fr/superieur/camDus2001.htm') . There was a call 
also already in 2002 

(http : //www. educnet. edu cation. fr/ superieur/campus2002 .htm') . We tried to open 
the Web sites of the Digital Campuses financed under the 2000 call for proposals, 
but unfortunately most part of the web sites were unreachable. 

Another interesting initiative is the Database maintained by the Observatoire des 
Ressources Multimedia en Education ORME, the body that monitors the use of 
multimedia resources in education (http ://www. orme-multimedia. orgA ). 

In the private sector there are an estimated 100 e-leaming companies in France. 
Although most French e-leaming companies were created only last year, some 
are actually traditional employee-training companies that have made the jump to 
the Net. 
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2.4 Italy 

Only three years ago Ana Dias (2000) wrote a description of the online education 
domain in Italy. 

”In 1990, the Ministry for the University and for Scientific Research and 
Development^^ promoted the creation of the NETTUNO network, for the 
implementation of Distance University courses. A consortium of universities, 
enterprises and other institutions composed this network. Politecnico de Milano, 
Politecnico di Torino, Universita di Roma, RAI, Trainet (a Telecom Italia 
Training company) all have largely contributed to the development of online 
education in Italy. 

Furthermore, several other institutions are enriching the online education field in 
Italy, schools are networking and providing online facilities like the "house of 
knowledge"^®, a collaborative and distributed learning environment designed for 
teachers and students. 

In a regional context there are also important experiences. The Emilia Romagna 
Region has put forward a plan designed by SINFORM^^ for the setting up of a 
"Resources Centre for Multimedia Education & Open Distance Learning". 
Today, the Resources Centre for the ODL is a reference point at regional level 
for experimentation and dissemination of training schemes based on the 
methodology of the ODL and is aimed at the utilisation of multimedia material. 
Besides national programmes and projects, European projects have largely 
contributed to the development of online education in Italy; face-to-face 
universities, companies, training centres are moving as fast as the information and 
communication technology demands.” 

Today things have evolved and the e-leaming market is alive. 

The conference ”e-Leaming: state of the art and future development' organised 
by ANEE^^ last April 2002, with the participation of the Ministry for Innovation 
and Technology (Ministero per ITnnovazione e le Tecnologie - MIT), concluded 
the following: 

- Three out of four enterprises are planning to invest in e-leaming solutions 
for training 

- E-leaming can not be planned within organisations without complete 
reorganisation of knowledge management 

- The E-leaming market is increasing 140% per year 

- Enterprises, Public Administration and Schools are the sectors in which e- 
Leaming will have a Strong impact 

- The Italian Government is heavily investing in e-leaming initiatives for the 
Public Administration both at central and peripheral level. 

- Important Italian Universities like the Politecnico di Milan or Bocconi are 
creating alliances with private Industry to develop common actions. These 
common actions go beyond a relationship customer-service provider since 
planning is done by both partner involved. 
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2.5 Greece 

Ana Dias (2000) wrote: ’’Greece, like the other Southern European countries, 
participates in different European pilot projects, but besides that, there is a low 
participation rate of Universities, Technical Education Institutes, Training 
Centres and Enterprises in the online education field. 

Located in Greece, the European Centre for the Development of Vocational 
Training (CEDEFOP) has been involved, since 1976, in the promotion and 
development of vocational training for young people, and the continuing training 
of adults, primarily through European-wide co-ordination of analysis and 
research activities. 

Today, this Centre represents the principal knowledge base on this subject, 
having published hundreds of reports and sponsored research in all Community 
member states. 

Within CEDEFOP, the European Electronic Training Village is a site dedicated 
to bringing experts in the field of Vocational Training together to share the latest 
information available. The electronic training village is an electronic resource 
centre (the users can download and read free publications, search lists on web 
sites by topic to find just the site you need, access bibliographical databases, the 
European Research Directory, the Terminological Database and the Institutional 
Database). In the words of CEDEFOP’s Director, Mr. Johan Van Rens, "The 
Electronic Training Village facilitates the flow of information amongst and 
between policy makers, researchers and practitioners in vocational education and 
training throughout Europe and beyond. Its aim is to stimulate communication, 
interaction and debate on the development of vocational training policy, practice 
and research. " 



Today the Greek School Network Ihttp ://www. sch.gr/index_en. phpL put the 
country's schoolchildren and educators in touch with each other and with resources 
all over the world. The schools are riding on the digital rails of the Greek Research 
and Technology Network (GRNET) Ihttp ://www. emet. gr/index_en.htmlL created 
under the auspices of the Ministry of Development's General Secretariat for 
Research and Technology Ihttp ://www. esrt . er/html/ eng/index. html I to interconnect 
Greece's academic and research community. 

Some good practices examples are being carried out among Universities and 
Scientific organisations, for instance at the Athens Medical School, a group of 
students take notes as they follow an operation being carried out at the Areteion 
hospital some kilometres away. The procedure is being broadcast live on an 
audiovisual screen in the central library of the National Technical University of 
Athens (NTUA) Ihttp://www.ntua. gr/en index.html , where a computer-sawy 
generation is reaping the multiple benefits of e-leaming. 

The NTUA in collaboration with Athens University Iwww.uoa.gr/home.htmI and 
the Athens School of Economics and Business Ihttp://www.aueb.gr/gb/main.htmn . 
has brought higher learning into the 21st century with the establishment of tele- 
teaching theatres at all three institutions. Professor Basil Maglaris, head of the e- 
leaming programme at NTUA, noted that e-leaming was especially useful for the 
medical field as practitioners are able to learn the latest techniques by actually 
viewing an operation online as well as offer tele-diagnosis services. Tele-diagnosis 
('http://tie.telemed.org^iblio/J is a crucial development for Greece considering the 
large number of communities living on remote Islands and in isolated mountainous 
areas. Furthermore, e-leaming is allowing students to come into contact with 
students and academics at institutions within Greece and around the world. 
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Main problem for the institutions is the lack of the financial support, which means 
that development of the site and its contents is an individual effort and 
consequently slow. 

We found that most of the websites of the institutions, universities, and firms were 
in Greek; this was a barrier to our investigation. In general we can say that there is 
just a small number of institutions applying real LMS systems, and also fi'om these 
reason there was just a small response on our project. 

From the small number of interested institutions there were some so busy that it 
was simply impossible for them to carry the interview even at the phone. 



3. Institutions and LMS experiences 

In the analysis presented in this chapter several types of institutions are analysed, 
fi'om Private Companies, to Enterprise Associations, Training Centres and 
Universities. The Institutions are presented by alphabetic order, within the 
countries Portugal, Spain, France, Italy and Greece. 

The institutions interviewed were ranging fi'om companies to non-profit 
organizations and Universities. 



Table 4. Institutions sorted by Country 



Name of Institution 


URL of Institution 


Country 


Type of Institution 


Academia Global 


www.academiaslobal.com 


Portugal 


Private Company 


Associagao Empresarial Portuguesa 


www.aeDortugal.Dt 


Portugal 


Private Non-Profit 

Association 


Associa?§o Portuguesa de 

Seguran^a Social 


www.seg-social.Dt/Drofisss 


Portugal 


Private Non-Profit 

Association 


Digito/Evolui 


www.evolui.com 


Portugal 


Private Company 


Instituto de Soldadura e Qualidade 


WWW. institutovirtual .Dt 


Portugal 


Non-Profit Private 

Institution 


Pt Inovagao 


www.Dtinovacao.Dt 


Portugal 


Private Company 


Universidade Catolica Portuguesa 


www.esb.ucD.Dt 


Portugal 


Private University 


Universidade de Aveiro 


www.ua.pt and 
www.unave.ua.Dt 


Portugal 


Public University 


Universidad de Vigo 


www.uvieo.es 


Spain 


University 


Anonymous 


www.esperienze . net 


Italy 


Anonymous 


Ifoa 


www.ifoa.it 


Italy 


Non Profit organisation 


Profingest 


www.profingest.it 


Italy 


Non Profit - Consortium 


Sinform 


www.odl.net/default.asD 


Italy 


Non Profit - Training 
Organisation 


Universita Cattolica del Sacro 
Cuore 


www3 .unicatt. it/unicattolica 


Italy 


University 


University of Trento 


www.didatticaonline.unitn.it 


Italy 


University 


CNED - Centre Nationale de 
Education a Distance 


www.cned.fr 


France 


Education/ Public 

Institution 


Universite de Bourgogne 


www.u-bourgogne.fr 


France 


University 


Universite La Sorboime Nouvelle 
Paris III 


www.tele3.net 


France 


University 


Universite Paul Valery - 
Montepillier III 


WWW. univ-montD3 .fr 


France 


University 


The Aristotle University of 
Thessaloniki 


www.csd.auth. er/information/d 
eoartment.en.Dhp 


Greece 


University 
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From table 4 we can observe a total of 3 private companies, 6 non -profit 
institutions (training organisations), 8 public universities, 1 private university, 1 
public institution in the education field and 1 anonymous institution. 

Do to its characteristics we have grouped all the public and private universities, 
together with the public education institution in a University group. 



Table 5 Type of organisation and frequency 



Type of organisation 


Frequency 


University 


10 


Private Companies 


3 


Non-Profit Institutions (training) 


6 


Anonymous 


1 



Table 6 bellow shows in detail the information about the surveyed institutions 
sorted by LMS used. Interviews include 11 self developed LMSs and 10 
commercially available LMSs. From the commercially available LMSs in use, 4 
Institutions use Web - CT (3 are Universities), 2 other Institutions use 
Blackboard, 2 other use Learning Space and one institution uses Docent and 
another institution uses Intraleam. 

Table 6 shows that 8 of the 21 interviewees institutions know or had experiences 
with other LMS besides the one in use currently. 

There are a total of 561 online courses that are using the 16 LMS systems. 
Important is to note that 4 institutions have fi’om 1 to 3 courses, 7 institutions have 
fi’om 4 to 15 courses and only 10 have fi’om 16 to 85 courses. These mean that 
50% of the researched institutions have less then 1 5 courses online. 

There are a total of 470 online tutors and a total of 41296 only students. Please 
note that teachers and students number is counted according to the number of 
years the system is in use (fi’om one year to seven years) and according to the 
duration of the courses (duration varies fi’om 25 minutes courses, to 30 - 60 hours 
courses, and 2-7 months, one semester, 2 years or 4 years courses). Another 
observation is that some institutions were not able to identify exactly how many 
only students (and some times teachers) they have. 

In summary, in the 21 institutions studied there are 16 LMSs in use (11 are self 
developed and 5 are i commercially available systems), 561 online courses, 470 
online tutors and 41.296 online students. 





Table 6. Information about Surveyed Institutions Sorted by LMS 
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Name of 

Institution 


LMS 


Other 

LMS 


# Online 
Courses 


U Online 
Tutors 


# Online 
Students 


# Years 
in Use 


Typical 

Course 

Duration 


Assoda^ao 

Empiesarial 

Portuguesa 


aep e-cursos 




30 


12 


500 


4 


30 hours 


Profingest 


Blackboard 5 




12 


26 


300 


2 


2 years 
MBA 
4 months 
(7 units 
courses) 


University la 
Cattolica del 

Sacro Cuore 


Blackboard 


Decus 

System 








2 




Sinform 


Docent 


Greentea 

m 


3 


11 


160 


2 


1 year 


University of 
Vigo ■ 


Elias 


Simulnet 


18 


38 


55 


2 


1 year 


Anonymous 


Esperienze 


Web-Ct 

Docent 

Intraleam 


1 


6 


150 


1 


4 months 


Digito/Evolui 


EvoluiTech 


- 


80 


45 


12500 


5 


1 month 


PT Inovapao 


Formare 


- 


10 


8 


1121 


6 


13 days 


Sinform 


Greenteam 


Docent 


7 


6 


140 


8 


2-7 

months 


Aristotles 
University of 
Thessaloniki 


In house developed 




1 




180 


1,5 


6 months 


CNED 


In house developed 


- 


- 


- 


- 


7 


Vary 


Academia 

Global 


Intralearn&Centra 


Docent 

Saba 


70 


15 


1500 


1 


25 

minutes 


IFOA 


L'aula virtual 




85 


50 


2200 


6 


2 years 
MBA 
40 hours - 
other 
courses 


Instituto de 

Soldadura e 

Qualidade 


Learning Space 


aep e- 

cursos 

Saba 


15 


12 


750 


2 


1-2 

months 


Universite de 
Bourgogne 


Learning Space 


- 


6 


3 


60 


1 


1 year 


University of 
Trento 


Proprietaiy 


- 


50 


10 


15000 


1 


1 semester 


Universidade 

Catolica 

Portuguesa 


TWT - teaching 
Web Toolkit 




14 


10 


120 


4 


40-60 

hours 


Associagao 
Portuguesa de 
Seguranga 
Social 


Web-Ct 




49 


6 


360 


3 


22-75 

hours 


Universidade de 
Aveiro 


Web-Ct 


Aulanet 


80 

University 
17 Unave 


80 

university 
11 Unave 


4500 

University 

500 

Unave 


5 


1-2 

months 

40- 

50hours 
and 80- 
100 hours 
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Name of 

Institution 


LMS 


Other 

LMS 


# Online 
Courses 


# Online 
Tutors 


U Online 
Students 


# Years 
in Use 


Typical 

Course 

Duration 


Universite la 
Soibonne 
Nouvelle Paris 

m 


Web-Ct 




3 


100 


1200 


1 


4 years 


Universite Paul 
Valery 

Montpellier III 


Web-Q 




10 


20 


10% of 
the 

University 

Students 


2 





4. Course Development Tools 

Course creation is generally observed as a main facility to the LMS. However, 
the answers to this part of the interview show different tendencies: i) LMS are 
accessible environments to course creation; ii) LMS are mainly a form of support 
and sharing of information; iii) LMS show difficulties which leads to the use of 
different or external tools and the involvement of production experts. A particular 
situation is observed in a self-developed system that is based in a flexible strategy 
that promotes the integration of new tools according the needs and the course 
design. 

LMS are accessible environments to course creation: 

”It is a simple and accessible environment. The content can be downloaded on the 
platform by the trainers, using any tool. Following a patterned structure fi-om the 
guide, the trainer can create a course, session to session and propose it to AEP” 
To some interviewees LMS are mainly a form of support and sharing of 
information: 

’’WebCT is a form of support, sharing information and communication. 

It is not seen as a system for the development of the course.” (Universidade de 
Aveiro; Web-CT) 

Some institutions need to use external tools to the LMS and specialist support to 
course production, also suggesting that difficulties are based not in the platform 
but in the process of implementation: 

’’INTRALEARN tool is somewhat debilitated and should be complemented with 
other tools and/or applications that generate content onto the web.” (Academia 
Global: IntraLeam) 

Some self-developed systems are based in a flexible approach to course design. 
”We could say that CNED has one basic system that allows new tools / elements 
to be added according to current needs, it is flexible, but problems always occur 
there. The creation of the courses is actually a joint work of CNED that provides 
know how, methodology, its expertise in distance education as such, in the work 
with various tools and on the other hand there is a institution e.g. university that 
import the content of the course. Usually when there is a course to be created the 
CNED examines if the tool of the institution could be useful or if the CNED 
should search for something new in order to provide according to best 
possibilities.” (CNED: Self-developed) 

On the whole the different platforms in use allow the didactic processes flexibility 
being rather, for some institutions, a main concern in the organization and 
development of the courses in the domain of the accessibility and interaction of 
the contents. One of them refers in particular to the adaptation of the forum to 
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the flexibility of the learning processes. For other institutions the integration in 
the various media is the means of didactic flexibility. 

’’The system used doesn’t impose or privilege any pedagogical approach. It’s a 
flexible system that allows different approaches. The focus is the possibility of 
designing activities with the students supported by the LMS.” 

The role of the forum to the development of didactic flexibility: 

’’The use of Web-Ct introduced a set of conditioning elements that had to be 
managed internally. 

For example, it was concluded that the chat didn’t bring an)thing of value to the 
pedagogical relationship with the students. 

The forum, on the other hand, became a set of reflective and advantageous 
discussions. Therefore, the material produced in the Forum became the 
pedagogical material and synthesis of material and themes. 

In this phase of creating material the software showed some inflexibility because 
it was necessary to have HTML in order to download the content/material for the 
e-leaming platform. Regarding didactic flexibility the software corresponded to 
the needs.” 

The importance of multimedia and audio and videoconference services: 

All formats are admissible as long as they’re, let’s say, video, audio, normal text, 
normal images, charts. The system has a description system in XML. The user 
has limits regarding suflBcient bandwidth, but the system itself doesn’t have any 
limitations. (Prodigio: EvoluiTech) 

In the teacher user-fiiendliness domain the answers point out different 
conceptions and strategies followed by the institutions. A former group refers 
that the platforms are clearly user-fnendly and also suggests that the design of the 
system will allow its intuitive usage, eventually with the help of helping systems, 
which promote the autonomy of the user. The second group establishes the 
necessity of trainers’ training to use LMS, and the third introduces the distinction 
between the roles of content expert and tutor, as a strategy to overcome the 
constraints of the familiarization with LMS. 

’’Very easy, the trainers that work with us have no specialty in computers; they’re 
traditional trainers, who have easily constructed the content without any 
problem.” 

”Our teachers who are also the tutors had no problems in using the LMS. For the 
most part they have found Blackboard to be very easy to use. In their opinion 
they can do more than before with our previous product. Valuable content 
development remains time consuming independently from what LMS you use.” 
The need of previous and/or continuing training: 

”It’s not perfect, a 4 on a scale from 1 to 5 ... there is no formal training for the 
teachers... the best practices is shared amongst the teachers, some workshops... 
Normally, a full day session, addressed to people with no experience as a user. 
The first part is an explanation of what has been done and a demonstration is 
given, and the second part is practical where the people use the tool. 

Afterwards, direct help is given to the teacher whenever he/she requires it... 

A team of students called techno-rangers was created, which works on a ‘call’ 
basis.” 

Some institutions introduce the distinction between the roles of the content 
expert (teacher) and the tutor, the later as the one that manage the course 
development and implementation: 
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”We distinguish content expert and tutor. There is no "Teacher” in a traditional 
sense. Content expert participate in course development following a top-down 
approach. They supervise content and reply to answers posed in the forum but all 
their input is mediated via the tutor. It is the tutor who uses the LMS directly not 
the content expert. 



The use of different media is a facility present in the generality of the platforms, 
in spite of its use not being usual, especially at the video level, having in mind the 
limitations of the pre-existing bands. 

It supports the integration of multimedia elements. Metadata for multimedia 
elements. 

Some institutions do not use audio or video but are planning to introduce them: 
”At the moment the LMS does not support audio, video or moving images but 
IFOA is analysing some existing products to be eventually integrated into the 
existing system.” 

Some institutions do not use audio and video: 

’’The system used is good for text mode, acceptable for images; it doesn’t 
support video or audio. Even though such resources are available for 
downloading, the system is not oriented to support audio or video in a broader 
sense.” 

Other institutions area cautious about the use of multimedia: 

”Yes it does, but at this moment we are not using audio and video in training, due 
to technical deficiencies in the students’ and country’s computers. The students 
don’t have computers with these characteristics, but the platform has it.” 

”No - we prefer not to load-up heavy content in order to facilitate those 
students who only have minimum software and hardware.” 



Questioning and assessment are facilities of the LMS used by the institutions of 
this study. However, these facilities have different ways to support student 
learning through the feedback offered to the user by the system/ tutor. One case 
of enriching feedback deserves a particular reference as it includes the 
development of a database of student’s interventions and comments presented as 
practical course applications. Another institution made the conceptual distinction 
between informal and formal assessment. The first one is included in the LMS 
and the later to the process of traditional in house final course examination. 

”It’s possible to create self-assessment questionnaires, as well as generate 
precedents. It is also possible to create work groups, generate questionnaires for 
final assessments without the student receiving any immediate feedback, given 
that it’s possible to create open questions.” 

’’The system includes a module for the creation and exploration of 
questionnaires with multiple answers. 

In a broader assessment context, it supports an approach by project, problem 
solving, exploration of virtual laboratories (field of chemistry, physics), not up 
for discussion, publication of work. ” 

Some LMS use external tools to provide questioning and assessment facilities: 

”We use automatic test batteries for formative and summative assessment. This 
software is not a Greenteam product but was integrated later. The product used 
is Academy. 

Other institution use a final course examination: 

”We have to distinguish between two types of assessment we use: informal 
assessment and formal assessment. The first is built in the course structure 
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through case studies to be developed by the students. The second one is never 
on-line but is done through traditional exams to be taken in house.” 

Another institution uses not only the tutor feedback but plan to introduce 
previous interventions and comments of students as a practical course 
application; 

’’The trainer has various question formats: multiple answer questions, 
correspondence questions, open-answer questions. Everything the user/student 
sends is corrected automatically, facing the correct answers stored in the server, 
and the system attributes a grade to the student right away. Everything that is an 
open-answer goes to the server area and the trainer checks every day the answers 
he/she must correct in order to give a grade. All open questions that require a 
jury or if that need to be evaluated by a trainer are sent by the trainer via web. 
That grade is then put together to the one given at the time in order to calculate 
the final grade. Another thing we are doing now is re-finding content fi-om the 
students... We save the interventions given by the students in each course, but 
we had never done anything with this before... We are now doing something..., 
as a way to enrich the courses, we are using the students’ interventions, 
descriptions made by them, exchange of comments, things of the sort... and we 
present it as practical course applications. . .” 



5. Student support tools 

The facilities of interaction available by LMS are shown, generally, by the fora, 
chat mailing lists and email, having to bear in mind that not all the interviewed 
institutions use the so called services. Some have adopted a pedagogical model 
which discourages the use of the services of chat communication. Others include 
videoconference and collaborative technologies based on video streaming. In 
one case, the strategy of development of the interaction still includes the 
promotion of the system control by the learner with the possibility of the learners 
being able to edit pages or even to create fora. 

Interactivity possibilities based in fora, chat and email services: 

The type of student support resource existent for communication is the forum, 
the chat room, the identification of a virtual class (that allows the sending and 
exchange of e-mails) and a document area (that allows the placement of 
documents for the virtual room). During training the students can share work 
documents or other documents that have to do with training, and it’s possible to 
not only send work or practical cases to the trainer but also to the virtual class, 
thus sharing work documents. 

Others also includes videoconference and video streaming technology: 

Various possibilities as chat, videoconferences, WebCT supports discussions. 

We integrated collaborative technologies (video streaming) using both life and 
traditional video lessons. The first connects the director and the orchestra. Email 
(all-tutor). Forum (all -tutor). 

Some institutions follow a pedagogical model that discourages the use of the 
synchronous communications: 

In fora. The institution doesn’t encourage the use of synchronous 
communication. Chats are not used fi’equently, synchronous tools are not 
promoted... the platform allows it, but they are not used often. 

One institution following an asynchronous strategy also encourage student 
control over the system: 



The students may ha\>e control over the system at various levels. They can be 



managers/creators of part of the course, and can create pages, fora, etc... but 
the basic idea is to participate in mailing lists, participate in fora, post content, 
publish pages... 



simultaneous usage of the communication synchronic and asynchronic services; 
ii) the preferential usage of the usage of asynchronic communication. Two cases 
are still referred of not using the communication services student to student. 

Both services of synchronous and asynchronous communication based in fora and 
chat: 

In synchronous terms it supports chat. There is no audio or video conference. In 
asynchronous and formal terms, it supports fora It should be noted that the 
management of iorzi, as well as other page models can be given to students. 

Some institutions develop the pedagogical model based in asynchronous 
communications 

It is mainly asynchronous communication although there are cases with the 
synchronise communication but it is quite rare. The CNED does use e-mails, 
chat, videoconferences, phone calls. It depends always on the student 
possibilities. 

Others are based in synchronous communication services: 

The ELIAS platform is synchronous, forum and e-mail is available. 

In the communication learner tutor two modes stand out globally, synchronic and 
asynchronic, respectively chat and forum, discussion lists and email. However, 
email is the modality more frequently stressed to enter in contact with the 
institution being still referred, in one case the discussion list for the contact with 
the institution. Theses services are available 24 hours a day. The availability for 
the communication learner tutor or institution includes the use of the telephone 
especially during the period of the activities. 

Online student to tutor/institution synchronous and asynchronous 

communications: 

Student and trainer communication is established in a synchronous and 
asynchronous way through four instruments (forum, chat, document zone, virtual 
room, questioning area). Regarding the institution e-mail is the only 
communication system in the platform. 

Or asynchronous communication, i.e. email, to student tutor/institution 
communication: 

email student - tutor. Tutors are available during the office hours. The presence 
of an additional technical tutor is guaranteed during videoconference lessons. For 
this purpose students have to join the class in a place which was equipped with all 
ITC tools required for effective video streaming. 

Other institutions use mailing lists to student institution communication 
The LMS privileges student to tutor communication. Institution communication 
has access to mailing lists by course, but the more administrative fluxes don 7 go 
through the system. The more administrative fluxes are implemented in a more 
traditional way. 

Some institutions use 24h telephone line as a communication resource 
Relatively to 24h access to the service, this is a reality, a permanent support 
telephone line functions beyond the normal working hours. 
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At the student to student communication level two tendencies stand out: i) the 
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Student facilities concerning library resources and online references are presented 
as follow: 

Access to online documents, references or a resource centre: 

At this moment there is no library on-line, the students have access to a set of 
documents that are made available in the work area of the course. 

Or an online library: 

There’s a Library space on the Platform. This library is general to all courses on 
the platform. By course, there is also an organisation of reference and resources. 
Links to online references: 

. ..the course has bibliographical references online and within each module there’s 
reference to books or publications that aren’t online. 

Some institutions also offer a course book. 

Documentation services are well organised, and have various services available 
on-line. 

Regarding UNAVE the basic work material is a commercial book. 

Other comments: 

..they are limited. But, we see it, not as a limitation of the system but of the 
resources. If the library had a digital service, this interlink would be natural and 
evident... we believe those services should be supplied by the libraries so they can 
be integrated by the LMS. 



From the answers given by the interviewed institutions we may consider two 
types of feedback: one is automatic and build up in the system and other is made 
by the tutor. The first type of feedback is immediate, since the student finish is 
work. However, there are few systems with this type of feedback facilities. The 
most usual form of feedback is given by tutors and varies fi’om 24h to the 
maximum of 7 days ( in a specific case of a doctorate course). 

Automatic feedback: 

Automatic feedback is provided by the system in reply to self-testing, and 
feedback coming fi’om the tutor is provided when the tracking system shows that 
the student does not make any progress. 

Time feedback that varies according the type of the course and the student tutor 
communication system used: 

It depends on each tutor/trainer... it is up to them... but that weakness exists at a 
presence level... there is no system that guarantees quality that can contribute to 
the fulfilment of the goals: time for response, level of use, quality of 

interactions, quality of activity design, etc... 

When we speak of e-leaming [Note Pedro Pimenta: e-leaming is understood as 
learning only supported by electronic means], these systems will be necessary... 
Or from 1 to 7 days: 

For a doctorate programme the time for response is acceptable. The teachers’ 
responses take from 1 to 7 days. The system does not advise the student. 
(Universidade de Vigo: Elias) 



6. Tutor Support tools 

There is evidence fi’om the data collected that not all LMS have facilities to 

monitor student’s performance, or when founded it is not enough for tutors’ task. 

The answers show that tracking facilities mostly address quantitative register of 

presence in the training area of the LMS, forum or chat. Also student 

administrative and background data is not directly accessible to tutors who need 

to made specific^query to the system administrator in order to get it. 

' » 
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Some interviewees identify facilities of student tracking with or the support of the 
system administrator. 

All trainers have access to their group of students, they have access to the 
identification of the students’ presence in the training area of the platform; they 
have access to what goes on in the chat room. The students’ tracking has to be 
solicited by the professor to the management of the training centre. 

To others tracking procedures are made by tutors: 

Tutors can track students via the LMS. Monitoring is mostly done through 
computer based assessment linked to specific databases. Personal data, test 
results and other forms of assessment get stored and are retrievable at any time. 

In some cases is used an external database associated with the LMS. 

Web-ct allows it, but the identification of the students is, in our case, done 
through a separate database. In the online tutoring mode monitoring is done by 
the tutors and by the course coordination. Tracking of students is done by the 
institution, which informs the tutors. 

Other LMS do not give tutor access to student data: 

The platform doesn’t have any monitoring tool. The teacher can analyse the 
answers to the evaluation questionnaires. The students’ data is stored in a 
database but the platform doesn’t support its manipulation. 

Others comments: 

This is not done through the LMS. This is the job of the institutes to monitor the 
situation, manage the contacts between all stakeholders, to be prepared to answer 
on any requests. They are responsible for registration that means they have the 
whole overview of the students’ portfolio, they usually receive the assignments 
from the students in order to see if they are fulfilling the required criteria for 
further studies. 

We get a few direct answers to the question related to group management tools 
(how the LMS deals , with the tutors facilities to manage group of students), 
especially to the group creation facilities. From the data collected management is 
made in the course of forum: However, there is a case of self-developed LMS 
where is identified an agent to guide group activities. 

With group creation and management facilities: 

The basic element is the group. Each course has a group. The teacher can add or 
remove students, send information, etc. . 

Without group creation or management facilities: 

There are no automatic facilities to make up groups of students. 

Group management facilities identified with fora, chat and email: 

Place tasks and communicate through e-mail with each work group. It can also 
launch specific fora. 

An agent to guide group activities. 

Teacher does not guide the groups but there is an ” Angel ” that helps the 
students by any problem. It works like the Angel appears and answers on the 
questions, always present and available; it is an internal part of the system. 

Mostly LMS have a questionnaire generator and it is used by tutors. However, 
some answers indicate that they are not user fnendly or accessible to tutors. To 
others situations questionnaires are prepared externally and displayed in fora. 
Questionnaire generator integrated in the LMS: 

There is the automatic generator of questionnaires in the platform. Regarding the 
availability of work and practical cases insertion is done by session. To insert 



Anal^is^ Paolo Dias, Pedro PUnentat Learning iVi^jiagenteni SystenisflMS) used in. Southern £urc^ 






session content there is a specific field in order to compose practical work on the 
case study. 

Some interviewees identify user fiiendly problems 

That is in the system but it’s not exactly given to the trainers because we think 
that they would not use it or would need a lot of preparation to use it... 
Questionnaires prepared externally and displayed in fora 

Questions can not be prepared automatically via some LMS specific publisher but 
are developed separately by the teacher. These test can be then downloaded by 
the student. The transfer of these files is done by the tutors via the tools provided 
by the platform. 

The collected data evidence a general lack of integrated tools to monitor and plan 
student progress. Exception was presented by two institutions whose LMS have 
dedicated tools to register and edit the student path. 

Otherwise, this function is accomplished through the questionnaires, assignments 
or, in one case, the working diary. Another pedagogical approach suggested by 
institutions to monitor students’ progress is based in the register of presence in 
fora. 

LMS with integrated tools to register student progress: 

It contains a set of progress reports. There’s the possibility to create various 
types of reports. The student’s whole path is registered. 

The tutor can edit progress by student or by course. 

LMS without specific integrated tools: 

The trainer doesn’t have access to automatic tools; he/she must ask management. 
The trainer receives the work directly from the students, but it has nothing to do 
with the platform. 

Through questionnaires and assignments: 

The evolution of the student can be monitored, regarding his/her progress in the 
foreseen training plan, at any time. 

Or fora activities: 

The tutors monitor through the answers given by the student in the fora. Through 
messages read and answered by the student. The register of pages accessed by 
the students. 



Data evidence the lack of specific tools to provide administrative communication 
between tutor and institution. Generally, administrative communication was done 
independently of the platform through external tools. Only one institution 
identifies these facilities in the LMS that they use, however without comments to 
the user-fiiendly level- of this tool. The answers evidence a current tendency to 
the absence of these tools in the LMS and the need to develop it in the next 
future. 

Some institutions identify tutor institution communication facilities in the LMS, 
although not used intensively: 

The LMS allows teacher/institution communication. For coordination a helpdesk 
is used, and normal electronic mail. 

Others do not have tutor institution communication facilities integrated in the 



LMS 

There is no specific part of the platform that makes tutors communicates with 
the: administration. Any communication of this kind is done ”out-side” the 
platform. 
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7 . Administration Aspects 

This section refers to general administration; students enrolment, fees payment, 
user and password management, student and examination records, and general 
teacher/student/course allocation. 



Enrolment procedures* and fee paying are made in quite different ways. The site / 
LMS of the institutions usually provide some information on the available 
courses, fees, etc... In this way, some preliminary contacts ("Pre-enrolment") 
might be facilitated by the LMS. In general, formal enrolment and fee paying is 
done outside the LMS. In some training companies or state services, enrolment 
might be made using the LMS, but payment is processed off-line (secretary, 
check, etc...). Only one of the institutions has declared to have the full process 
supported by the LMS. 

There are contradictory expectancies; although, in some way (... still, lacking , 
not yet, it is planned . . .), some users mentioned the fact the payment should be / 
will be supported by the LMS, one other clearly stressed that the LMS should 
focus on teaching/leaming activities, considering that administrative functions are 
out of the scope of the LMSs. 



Concerning Passwords and security, in general, access to course content is 
granted based on login / password. Some LMS support both public and private 
areas. No one related any security problem. One person mentioned that the 
security strength is related to the operating system used. Some systems use a 
'user' approach, and the usemame/password are provided to the 'student' / 'tutor'; 
one other mentioned that the password is associated to the course. 



Although all the LMS claim to have, in some extent, a students database, the use 
is quite different; in some cases the database is not available or the institution "is 
not ready to use" it. Other situations includes tracking of the students behavior 
and/or quiz (closed-type questions) performance inside the system. One system 
has different databases for students and for tutors and teaching staff 
Some answers pointed out that users are, in same way, disappointed with the 
database extraction services. 



"Examination", "evaluation", "formal assessment" are understood as formal 
processes, and the face to face model is followed. On-line students might be 
submitted to "formative" and/or "summative" assessments, for 'monitoring 
learning' purposes, and the issue of "participation certificates", based on students' 
activity inside the LMS is a generalized procedure. 



Courses, teachers, classes is a 'design' subject. Some LMS have been designed to 
have some tutor/classes managerial facilities, some others don't. In the first case 
the LMS provides some 'scheduling' facilities, in the others the 'working 
perspective' is the course unit. 

It seems a general trend that this facilities seem much more important for 
professional training institutions where (in general) courses are shorter, are 
"repeated", in several "editions", in short timelines, than in classical education 
institutions (universities), where the model adopted (longer, once a year) favors a 
more stable course / teacher / students association. 
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8. Technology Aspects 

From the point of view of hw/sw, encountered systems seems to reflect the 
overall market situation; the majority of systems use Microsoft software 
(Windows 2000, NT), but some systems use Unix / Linux as the operating 
system. For traditional teaching institutions adopting LMS, the integration with 
other information systems is not a present concern; for institutions basing all their 
activity on the LMS the integration is already obtained or is considered 
prioritaire. 



One interesting point is the fact that two interviews didn't knew what kind of 
hw/sw their LMS were using. This might be understood a positive tendency in 
terms of the opacity of technology; people are using technology without 
necessarily be aware of what kind of alternatives are being used, which might be a 
sign of the maturity ofithe technology. 



A main concern of the eLearning providers seems to be the easy of use for the 
final user. All the answers mentioned the idea of 'minimum requirements' for 
Client hw/sw, using expressions as "the client only needs... ", or "standard 
equipment... ", "basic programs", "minimum user requirements", etc. . . 
Concerning the hardware, some interviewees mentioned CD player and printers 
(beyond the 'standard'); in what concerns the software it is possible to separate it) 
the access to and navigation into the LMS and ii) the access to course contents; 
for i) 'standard' browsers are usually mentioned; for ii) several interviewees 
mentioned the necessity of having some plug-ins, being Acrobat explicitly 
mentioned. 



One particular answer mentioned some other type of requisites; no hardware 
neither software, the need of an email account has been mentioned as a 'client' 
requisite. 



The question of the flexibility of didactic structure got a very broad range of 
answers; some interviewees mentioned the fact of using or complying with 
standards (IMS, SCORM), others (probably because 'Flexibility' is seen as a 
positive thing, by itself), argued that 'the system is new', or, in a very laconic way, 
"It is adaptable". Many interviewees argued that technological design and 
development had taken in consideration the importance of pedagogical freedom 
pedagogical ... adaptation", "pedagogical strategy"). 



In relation to Standards, interviewees stressed the absence of both 'de facto' and 
'proposed', technical standards. Curiously, (only) one interviewee mentioned 
his/her belief that standardization will have a positive impact in 
internationalization of eLearning businesses; and, on the other hand, other person 
stated the complementary idea "Since our courses are country specific, standards are 
not yet relevant". The existence of standards is welcome, both for marketing 
reasons (as indicated before), but also for cost reduction ('rationalization of 
resources') or LMS migration. 



The number of students is not considered an issue; many interviewees stressed 
the fact that their experience is relatively recent, or the number of students quite 
small; others mentioned systems is scalable. Only one person mentioned the fact 
of having one server for course contents management and delivery and another 
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for users' management. Another person mentioned the use of two servers, but for 
security reasons. 

Some phrasing indicates that interviewees are aware that number of students 
might be a problem in the future; "we are just at the beginning(...) no problems 
yet", "no problem until now", "just a small number of students ... yet", "up to 
now...", and, in some way, in are foreseeing the question. 

In general, access to the LMS is through Internet, and the speed of the system 
depends on the limits of bandwidth available for each user. Users are accepting 
this situation as satisfactory, except two interviewees; one uses video contents on 
their courses, and the other has a large experience in corporations with their own 
network. 



9. Economic Issues 

Interviewees have been conducted in a broad range of institutions; from 
Universities to professional training companies, and the answers on these 
questions reflect this diversity. Given the fast evolution of these products, and the 
correspondent fast obsolence, providers prefer a cost structure based on annual 
fees than based of 'price'. Commerci^ and/or well-established LMS have 
substantial costs, and again here two main cost structures are present; one-year 
license ranging from $5000^ to 40000/60000 €, independent of the number of 
users, or licenses based on the number of users, with a cost of 30 € for each 
student. The institutions which have developed or are developing their own LMS 
don't mention the 'cost', or express the developing / maintenance costs in terms of 
people allocated (1 technician full time). One University is using a open source 
product with buying cost zero. 



Student fees issue relates directly to the course undertaken, not the platform 
used. Companies working in the area of short time, vocational training, 
mentioned student fees in the range 175 €-200 €. One University, which uses an 
LMS for doctoral programs, mentioned a initial fee of 1250€ plus 800 € per year. 
In the cases where the LMS has a cost per user, the institutions include it on the 
course price of - mainly Universities - offer the LMS as complementary service to 
teachers and students, without any fee directly linked to the LMS use. 



Many interviewees mentioned that is hard to identify the staff support and costs. 
However, those who gave objective values, converged for a team of around 2~3 
people full-time (technicians plus help desk), plus a variable size support team 
depending on the number of users who need help. 

Teachers / tutors usually require some initial training in using LMS. In some 
cases teachers are well acquainted with technology, and formal training does not 
take place, in other cases, formal actions are planned, and some interviewees 
mentioned blended learning practices, as one day face-to-face plus 15 days at 
distance. Some interviewees stressed the fact the people needing support in 
learning how to work with LMS are teachers, since the LMS interface for 
students follows the web standards, and, thus, students don't need help in using 
the LMS. In the case of blended learning, the first, face-to-face session, is used by 
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the teachers / tutors to show to the students the functionalities of the LMS, and 
this is considered enough for the rest of the course activities. 

10. Overall evaluation 

Interviews analysis led us to find that in general the e-Leaming managers are 
satisfied with the solutions they bought or developed. 

All Web-Ct users declared they were satisfied with Web-CT. One has pointed out 
difficulties with the English language and another considered the interface very 
poor. 

Blackboard users are satisfied and hope the Italian version will come out to the 
market very soon. 

Learning Space is considered good, stable and intuitive. Interviewees pointed out 
that the assessment aspect is not very well constructed. 

Some of Docent users would appreciate that the handbooks was not in English, 
but translated. Furthermore the graphical impact was considered of Docent is not 
very attractive. 

Intraleam users considered it a program of good quality and flexible. They stated 
that the system was better than Learning Space or Formare, but worse than 
Docent or SABA. 

Concerning the self developed systems the majority of the interviewees are 
satisfied with own systems. Most part of the interviewees were able to point out 
some major difficulties and some improvements they would like to introduce in 
their own developed systems. 

Some own developed platforms are very successful and are selling e-Leaming 
solutions to other Institutions. 

"...it is a national system, developed according to the practice and experience 
of e-learning, maybe that can be considered the secret to its user-friendliness 
and success in the increase of the number of clients in 2001 and 2002 (currently 
there are 17 institutions using the system in Portugal). ” 

Other own developed systems are satisfied with their LMS features, but they are 
not able to put Multimedia on the web due to the general technological network. 
In relation to trainers it was stated that trainers can put all kinds of contents in the 
system, but possibly the LMS could integrate some other trainers facilities, 
especially related to students monitoring. 

One University that developed its own system declared that there are other 
factors that limit the use of LMS; 

"There is no institutional strategy, it is a bottom-up initiative, but it would be 
useful if it were top-down... the initiative is facilitated and accepted, but there is 
no strategic dimension...” 

Another University supports previous declarations: 

’’The experience is good and surpasses the expectations. The most important is 
lacking. . . The institution dedicates very little to virtual education. The only thing 
that the University acknowledges is the dedication of the teacher, counting it as a 
school year activity and thus valid for curricular reasons.” 

In summary we can say that Commercially available platforms can be very 
practical to start with but they have problems with linguistic issues, as well as 
with assessment tools adequacy to target groups. 

Own developed systems are simpler and directly related to the target groups; they 
surpass the linguistic problems of the commercially available platforms and are 
constantly updated, being able to improve their features according to trainers, 
trainees and administration evolution. Besides the linguistic advantage national 
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marketing strategies together with competitive pricing contribute to the great use 
of those own developed LMSs. 

Another important issue is the Universities e-Leaming managers concern with the 
University policy and strategies for this field. Apparently Southern European 
Universities are note dedicating enough importance and attention to this subject. 

11, Future Features to include in LMSs 

Content management and content development tools are major concern for all 
interviewees. 

"Enrich content: multimedia, and knowledge management. Content flexibility. " 



"It’s necessary to improve the content management databases. There is software 
that only manages content that can be incorporated " 

A general concern for the e-Leaming managers is related with the management of 
competencies, including the integration between training management and human 
resources management. Interviewees declared also their concern with the 
administration and monitoring of students, courses and contents, as well as with 
the use of better assessment tools. 

"Integrated management of all management/organisation mechanisms of the 
course, mainly in a dematerialised environment of a course, in the perspective of 
the system administrator, so as to allow a global reading of the course. 
Integrated management system capable of monitoring. ” 



"Students data bases, further group management tools to better divide users, 
better articulated statistics (one per class, alphabetic order); the possibility to 
work directly on course development without the need of using dream river. " 



Other institutions have great concerns with the Teachers training. 



Didactic flexibility and online retrieval of library resources are also concerns 
pointed out by interviewees. 



”.. without losing the current perspective of flexibility, it would be interesting to 
have pre-defined models of structured pedagogy... a case of methods, a problem- 
based learning, or other type of approach that has already been structured, etc... 



"Integration with other systems; Support of different media... Standardization for 
the reuse of contents/activities... more in the perspective of activities... 



Many institutions did not answer to this question, especially institutions fi"om 
France, none of them have answered. 



In summary, content management, didactic flexibility issues, including 
collaboration and group management and students management are the main 
concerns expressed on the interviews. 



57 



Alia Dias, Paolo Dias, Pedro Pimefttai Learning Management Systems <LMS) iised in Southern Enr<^ 



12. Conclusions 



From the present research it is clear that the increasing number of Internet users 
in Southern Europe is pushing up the e-Leaming market. There are more 
institutions, which have web presence and e-leaming offer. 

Besides the eLearning initiative of the European Commission seeks to mobilise 
the educational and cultural communities, as well as the economic and social 
players in Europe, in ; order to speed up changes in the education and training 
systems for Europe's riiove to a knowledge-based society. 

Viviane Reding, Commissioner for Education and Culture stated 
"The Member States of the European Union have decided to work together to 
harmonise their policies in the field of educational technology and share their 
experience. eLearning aims to support and coordinate their efforts and to 
accelerate the adaptation of education and training systems in Europe." 

It is our conclusion fi-om the present studies that Southern European institutions 
are on the write track to further develop the existing e-Leaming offers. 

Since the year 1998 we have observed the e-leaming field evolution. Ana Dias 
(2000) wrote in the Cisaer final report: 

” Online education in Portugal is a very unstmctured domain, at both public and 
private level. In almost all analysed cases, the web courses provided are 
developed in the context of pilot projects, financed by the European Union and 
with a limited hfe span.” 

"Online education evolved rapidly in Spain and there are a large amount of face- 
to-face universities, training centres and private companies adopting this type of 
learning.” 

” In France, like in the rest of Europe, European pilot projects are responsible for 
a large part of the market movements.” 

"Besides national programmes and projects, European projects have largely 
contributed to the development of onUne education in Italy; ” 

"Greece, like the other Southern European countries, participates in different 
European pilot projects, but besides that, there is a low participation rate of 
Universities, Technical Education Institutes, Training Centres and Enterprises in 
the online education field. ” 

The present study shows evidences of an evolution of the institutions involved in e- 
Leaming in Southern Europe. 

The pilot projects are no longer dominating the e-Leaming field in Southern 
Europe. 

At present there is a tendency to organise and stmcture the e-Leaming offer using 
a type of software somewhere classified has Learning Management System. Those 
systems are dedicated to some issues of the learning process, but in almost all the 
cases (commercially available systems or self developed) the systems are not able 
to perform all the activities the institutions need. Administration aspects, 
integration with existing software and content management are some of the issues 
not well treated by most part of the LMS studied. 

Language is a main issue in Southern Europe and LMSs not translated to 
Countries languages can be easily unsuccessful. 

Another interesting observation is the lake of a common understanding concerning 
terms and functionalities of the LMS systems. 
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The research also led us to observe that most part of the e-Leaming managers 
assume a position of experimentation and initiation on the e-Leaming process 
(50% of the researched institutions have less then 15 courses online). 



Course creation is generally observed as a main facility to the LMS. However, 
the answers to this part of the interview show different tendencies; i) LMS are 
accessible environments to course creation; ii) LMS are mainly a form of support 
and sharing of information; iii) LMS show difficulties which leads to the use of 
different or external tools and the involvement of production experts. 

To some interviewees LMS are mainly a form of support and sharing of 
information: 

Some institutions need to use external tools to the LMS and specialist support to 
course production, also suggesting that difficulties are based not in the platform 
but in the process of implementation. 

The student support tools available on the LMS are shown, generally, by the fora, 
chat mailing lists and email, having to bear in mind that not all the interviewed 
institutions use the so called services. Some have adopted a pedagogical model 
which discourages the use of the services of chat communication. Others include 
videoconference and collaborative technologies based on video streaming. 



There is evidence from the data collected that not all LMS have facilities to 
monitor student’s performance, or when founded it is not enough for tutors’ task. 
Also student administrative and background data is not directly accessible to 
tutors who need to make specific queries to the system administrator in order to 
get it. Some interviewees identify facilities of student tracking with the support of 
the system administrator. 

These observations lead us to conclude that in most part of the cases Teachers 
are note the ones who monitor the students. 



It seems a general trend that the Administration facilities seem much more 
important for professional training institutions where (in general) courses are 
shorter, are "repeated", in several "editions", in short timelines, than in classical 
education institutions (universities), where the model adopted (longer, once a 
year) favors a more stable course / teacher / students association. 



Concerning the technological aspects, in general, the access to the LMS is through 
Internet, and the speed of the system depends on the limits of bandwidth available 
for each user. Users are accepting this situation as satisfactory, except two 
interviewees; one use$ video contents on their courses, and the other has a large 
experience in corporations with their own network. 



Many interviewees mentioned that Economic aspects are hard to identify. 
However, those who gave objective values, converged for a staff team of around 
2~3 people full-time (technicians plus help desk), plus a variable size support 
team depending on the number of users who need help. Teachers / tutors are also 
an additional cost, they usually require some initial training in using the LMS. 

An overall evaluation allow us to observe that the commercially available 
platforms can be very practical to start with but they have problems with 
linguistic issues, as well as with assessment tools adequacy to target groups and 
pricing. f- ^ 
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Own developed systenis are simpler and directly related to the target groups; they 
surpass the linguistic problems of the commercially available platforms and are 
constantly updated, being able to improve their features according to trainers, 
trainees and administration evolution. Besides the linguistic advantage national 
marketing strategies together with competitive pricing contribute to the great use 
of those own developed LMSs. 

Another important issue is the Universities e-Leaming managers concern with the 
University policy and strategies for this field. Apparently Southern European 
Universities are note dedicating enough importance and attention to this subject. 

Most part of the systems researched seams to have problems with content 
creation and content management, students monitoring and assessment tools. 
Online administration and integration with other institution software and 
platforms were also a question. 
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The use of Learning Management Systems in North Western 

Europe 

by 

Desmond Keegan 



Overview 

This report is an analysis of in-depth interviews with systems managers of Learning 
Management Systems (LMSs) in North Western Europe: 

Ireland 8 

Great Britain 6 

Northern Ireland 4 



Country 


Language 


Geographical Area 
(sQ.km.) 


Population 

(millions) 


Ireland 


English 


70282 


3.6 


Great Britain 


English 


227480 


57.6 


Northern Ireland 


English 


14120 


1.6 


Total 


English 


311882 


62.8 



Table l.Demographical data on North Western Europe 




This study focuses on the satisfaction or lack of satisfaction of institutions with the 
LMSs they have purchased or developed themselves. In the areas under 
consideration the term Virtual Learning Environment (VLE) is used more 
extensively than Learning Management System (LMS). 

Background 



The phenomenom of education and training on the World Wide Web is relatively 
recent. Collis, in her influential 1996 volume, Tele-leaming in a digital world: the 
future of distance learning, places the first courses in late 1995. 



After a first exploratory period the first LMSs started to appear and listings of 
them today by Brandon-Hall in the US and by Bruce Landon in Canada reach to 
over 100. Many of the LMSs presented in this study are not listed by either author. 



Here is a generally-accepted diagram presentation of an LMS: 
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environment) 

Brandon-hall.com defines an LMS as "software that automates the administration 
of training events. An LMS registers users, tracks courses in a catalogue, and 
records data from learners; it also provides reports to management. An LMS is 
typically designed to handle courses by multiple publishers and multiple providers." 

LMSs "allow a company to leverage the collective knowledge and skills of its 
workforce more strategically," according to WR Hambrecht and Co’s report 2001 
Outlook for the Learning Management System Market. "In addition, an LMS 

• provides the foundation to improve the speed and effectiveness of the training 
process, 

• ensures that an enterprise is in compliance with relevant industry education 
standards 

• enhances the efficiency of a company’s supply chain through better product 
knowledge, and 

• improves communication among and retention of employees during a business 
transformation process." 

American forecasts calculate that about 60 percent of corporations will have an 
LMS platform deployed by 2003. 

Selecting an LMS isn’t something to be taken lightly. In all likelihood, the task will 
be lengthy, intricate, and costly. Considering the fact that most analysts agree that 
the life span of a typical LMS is only two years, it’s crucial to be able to 
demonstrate the return on investment. WR Hambrecht and Co concludes that 
purchasing an e-leaming infrastructure "involves a long decision-making process, 
extensive custom programming, and time-consuming installations. Our 
conversations with buyers of LMS products indicate that the high cost of switching 
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LMSs makes customers more loyal even when they’re dissatisfied with certain 
elements of their LMS." 

Here are the main qualities most current LMSs have, according to brandon- 
hall. corn’s report Learning Management Systems 2001: Hew to Choose the Right 
System for Your Organization -. 

Browser-based applications. Most of the new systems are 100 percent Web-based 
and use Java and server applets. The vast majority of LMSs let administrators and 
learners access performance data and reports using only a browser. 

Authoring tools. Many systems offer Web-based, built-in authoring tools geared 
toward subject matter experts with authoring skills. Systems that don’t have built- 
in authoring tools are designed to use standard, off-the-shelf tools such as Flash, 
Authorware, Dreamweaver, and ToolBook. 



Assessment tools. Many LMS products offer tools to create tests and assessments. 
The data produced by the tools tends to be more system-compatible than data 
created by third-party applications. Similarly, 45 percent of the tools provide skill- 
gap analysis, showing an increasing trend toward using LMSs to monitor overall 
human performance. 



Blended-leaming capabilities. Classroom and e-leaming management features 
such as classroom scheduling, enrollment, and wait listing are now included in 
most offerings. 

Compliance with industry standards. Support of e-leaming standards has become 
a priority among LMS suppliers. Fifty-two percent of products currently support 
Aviation Industry Computer-based training Committee standards; 75 percent 
indicate they’ll support AICC; and 59 percent say they’ll support metadata tagging 
specifications by 2002: 



Larger implementations. The number of learners has grown substantially in the 
past four years. Suppliers say that the number of learners in their largest individual 
implementations is about 116,000, compared with approximately 39,000 learners in 



1997. 



Increased cost. Data shows that the cost of LMSs is risingdramatically. The 
average cost of a system supporting 8,000 learners over a five-year period is 
currently US$550,000. 



Use of LMSs 

The Learning Management Systems (LMSs) used by the institutions in this study 
were: 



WebCT 

TopClass 

Blackboard 

Ascot Systems Course Master 

IT Campus 

Intranets 

Visit 
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6 

3 

3 

1 

1 

1 

1 
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Granada Leamwise 
FDL Learning Environment 



1 

1 



Table 2 lists the institutions of which the LMS System Manager was interviewed. 
Interviews were conducted with institutions in Finland and Italy as well, but these 
are not analysed here. , 



Institution 


URL of Institution 


Country 


Type of institution 


DEIS, Cork IT 


www.cit.ie 


Ireland 


Institute of Tech 


FAS 


www.fas- 

netcolleee.com 


Ireland 


Govt. Training 
Centre 


Dublin IT 


www.dit.ie 


Ireland 


Institute of Tech 


Sligo IT 


www.itslieo.ie 


Ireland 


Institute of Tech 


Cork IT 


www.cit.ie 


Ireland 


Institute of Tech 


Manchester 
Metropolitan Uni 


WWW. Itu. mmu. ac.uk 


Great Britain 


University 


CREATE 


Create. sufiFolk.ac.uk 


Great Britain 


University College 


Uni of Ulster 


www.ulster.ac.uk 


Northern Ireland 


University 


Oaklands College 


www.oaklands.ac.uk 


Great Britain 


Further Education 
Coll 


North Eastern 
Institute of F and H 
Ed 


www.nei.ac.uk 


Northern Ireland 


College of Further & 
Higher Education 


City College 
Manchester 


www.ccm.ac.uk 


Great Britain 


Further Education 
Coll 


Reading Coll of Arts 
& Design 


www.readine- 

cblleee.ac.uk 


Great Britain 


Further Education 
Coll 


University of Galway 


www.openleaminecen 

tre.com 


Ireland 


University 


Dun Laoghaire Inst of 
Art, Design and Tech 


www.iadt-dl.ie 


Ireland 


Institute of 
Technology 


Upper Bann Inst 


www.ubi.ac.uk 


Northern Ireland 


F and HE Coll 


Belfast Institute of F 
&HEd 


www.bbelfastinstitute. 

ac.uk 


Northern Ireland 


Further & Higher 
Education Coll 


Unique Learning 


www.uniqueleamine. 

com 


Ireland 


Company 


Tameside College 


www.tameside.ac.uk 


Great Britain 


Further Education 
Coll 



Table 2. List of institutions in study 



An immediate conclusion from the study is that the countries of North Western 
Europe are the English-speaking regions of Europe. In the use of LMSs for e- 
Leaming there is an immediate dependence on the major American LMSs, 
especially WebCT and Blackboard, who are the market leaders. 

In the last few years there has been extensive promotion of WebCT, and to a lesser 
extent Blackboard, to Irish and British colleges and universities. WebCT has 
embarked on a concerted sales drive to convince the British and Irish higher 
education sector to adopt WebCT. 

This confident approach of WebCT to the British and Irish higher education 
market is mirrored by press releases from March 2002 about the economic success 
of WebCT in the depressed Information Technology market of this period: 

WebCT , the world's leading provider of integrated e-leaming systems for higher 
education, has announced record financial results for the year ending December 
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31, 2001. The highlight for the privately held WebCT was a substantial increase 
in total revenue: 2001 total revenue increased 292 percent year-over-year. 

The Company also announced triple-digit year-over-year increases in every 
significant revenue category, including license revenue, which rose 334%; 
publisher content revenue, up 869%; and sales and support revenue, which 
increased 158%. WebCTs record 2001 financial results underscore its continued 
dominance of the higher education e-leaming category. 



Two decisive factors contributed to WebCTs explosive revenue growth in 2001. 
First, the company continued to add new higher education customers at a record 
pace, winning an extraordinary 72 percent of all competitive new business 
evaluations in the past year. As a result the Company grew its customer base by 
more than 40%, adding more than 750 new institutions to its global customer 
network. WebCT, which already had the world's largest installed customer base in 
higher education, now has a course management system network of more than 
2500 institutions in 81 countries around the world 



Internationally, WebCTs 2001 customer acquisition was particularly strong: the 
company signed on over 500 new international customers in 2001 alone. This 
growth was fueled by the company's continued expansion of the localization 
program of its platforms in 10 major world Icmguages, and the increase of its 
global reseller network. 



At the same time that WebCT grew its customer base in 2001, the company also 
successfully expanded existing customer relationships, thanks in large part to the 
successful rollout of WebCT Campus Edition. In the first nine months of its 
availability, WebCT has over 180 campuses that have purchased its Campus 
Edition software product. The majority of these sales came from existing customer 
upgrades, reflecting the strong commitment WebCT's customers have made to 
supporting their institutional goals of improving academic performance and access 
through their campus' course management system. 



"Our significant revenue and customer growth in 2001 demonstrates the power of 
WebCT's business model, which is focused exclusively on e-leaming," said Carol 
A.Vallone, president and chief executive officer of WebCT. "Building on the 
momentum of last year, we are excited to continue to lead the higher education e - 
learning industry, as we prepare to deliver a tmly breakthrough product that will 
enable institutions to achieve their goals, and take e-leaming to a whole new level." 
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Table 3 presents data on the use of LMSs by the institutions iaterviewed: 



Name of 
institution 


LMS 


Other LMSs 
used 


# 

online 

courses 


# online 
tutors 


# online 
students 


# years 
in use 


Typical 

course 

length 


DEIS, Cork 
IT 


Top Class 
3.1.1 and 
5.0 


Proto, 

Blackboard 


10 


3 

fulltime, 

others 


400-500 


Since 

1995 


100 horns 


FAS 


TopClass 


Skills Vantage 


- 


2 


2100 


4 


6 months 


Dublin TT 


WebCT 


TopClass, 

Intranets 


12 


- 


100 


2 


varies 


Sligo IT 


WebCT 


Blackboard 


2 


3 


60 


0.5 


3 months 


Colic IT 


TopClass 


Moving to 
WebCT 


- 


- 


- 


Since 

1995 


- 


Manchester 
Metro Univ 


WebCT 


In house 


150 


190 


7000 


4 


2-120 hours 


CREATE 


WebCT 


Solstra Hybrid 


13 


9 


800 


18 

months 


5-30 hours 


University of 
Ulster 


WebQ : 


TopClass, 
Blackboard, 
First Class 


150 


400+ 


24.000 


2 


semester 


North East 
Inst of F & 
HE 


Ascot 

Systems 

Coursemaste 

r 


Net tutor 












Upper Bann 
Institute 


IT Campus 


Plato, 

Destinations 


50 


120 


4000 


1 


34 weeks 


Belfast Inst 


Blackboard 


None 


139 


- 


1000+ 


2 


various 


U of Galway 


Intranets 


None 


1 


4 


22 


1 


2 years 


Dun 

Laoghaire 

lADT 


Blackboard 


WebCT 


6 


4 


140 


4 


3 years 


Unique Learn 


VISIT 


Leam2 


1200 


- 


- 


1 


6 months/ 1 



Oaklands 

College 


Granada 

Leamwise 


TekniCAL 
Virtual Campus 


- 


3 


- 


1 


various 


City College 
Manchester 


Blackboard 


none 


20 


100+ 


1000s 


1 


variable 


Reading 

College 


FDL 

Learning 

Environmen 

t 


none 


4 


6 


60 


7 

months 




Tameside 

College 


WebQ 


none 


50 


400 


17.000 


1.5 


1-50 hours 



Table 3. Data on use of LMSs by institutions 
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The types of institutions in the study were; 



Universities 4 

Institutes of Technology 5 

CoUegess of higher and further education 7 

Government training agency 1 

Training organisation 1 



Course creation 



Distance education systems are usually held to have two complementary 
subsystems: course creation and student support services. These two subsystems 
are both necessary to distinguish distance systems from Teach Yourself Books and 
to justify the award of university degrees, college diplomas and training 
certification for courses studied at a distance. In electronic learning (e-Leaming) 
these two subsystems reappear. 



WebCT was ranked from 'excellent' to 'very good' by those who used it. TopClass 
was cited as having a medium level of difSculty for use and 'not so good' by 
another. 



Typical comments were; 



We have found WebCt to be a pedagogically neutral tool, i.e. it does not impose 
a particular learning style on staff / students. We have examples of didactic and 
constructivist courses along with communication rich learning communities. 

The current WebCT designer interface does require some staff development 
training. However, this tends to be done along with the necessary instructional 
design training. 



The instructor interface is intuitive for new users to use with minimal training. 

WebCT provides a media library facility and a means of integrating CDRom 
based content. WebCT provides self test, quiz and assignment tools. The quiz tool 
comes with comprehensive reporting facility. 



And again: The WebCT structure provides a framework which does not impose 
a particular pedagogy. The system requires a certain level of IT skill, mainly to 
do with file management and windows-type operations. I reckon initial training 
takes about two days. It is fully multi-media compatible. 



Blackboard was judged as being excellent, fast and simple for course creation. It 
was considered extremely easy for use by teachers and course developers. 

Blackboard was considered to allow complete didactic flexibility for users. It was 
considered very user fiiendly. Tutors like the interface and the ease of use. It does 
not require high technical prowess. 
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Top Class was ranked as: Medium level difficulty for course creation. At times it 
is counter intuitive, even when one uses the Top Class Publisher tool, 

and 

Top Class provides a publishing suite. Using a product called TopClass Assistant 
you can convert a Word or (less successfully) a PowerPoint file into a plug file 
which can then be uploaded to the server. From Word or PowerPoint you can 
also create a TopClass Publisher file which allows for easy-editing of structure 
and content before you again publish as a plug file and upload Publisher can 
also be used to create courses from scratch. You can also write individual pages 
in HTML live. Publisher doesn't work as it is meant to: All kinds of problems 
occur in the conversion of Word to Plug, or Word to Publisher or (less frequently) 
Publisher to Plug files. 

The didactic flexibility of WebCT was acknowledged, but TopClass was 
considered too structured and templated. The reason for this was analysed as the 
concept of a course in the TopClass system: 

A major factor is the way it understands what a course is. It forces you to regard 
a course as a series of folders and quizzes and bundles them together as a course. 
It is sufficiently open-ended to allow you to use discussion forums, case 
conferencing, web weaving to expand this in-build notion of a course. It provides 
these and you can try to make them work towards creating more innovative 
interactive courses. 

On the question of teacher userfriendliness WebCT was considered very 
userfriendly and easy to use. Top Class demanded 'a long learning curve', 
calculated at several months: 

It was my intention to learn it on my own but it was a long learning curve - it took 
several months. 

Support for graphics and moving image was considered satisfactory in WebCT. 
TopClass was also acceptable except for the creation of batches of mathematical 
equations. 

Questioning and assessment provision was acceptable in WebCT which was 
considered to provide a 'variety of options'. TopClass was criticised for providing 
only quizzes and these were not considered relevant for university teaching: 

There is plenty of multiple choice/Boolean/true or false questioning techniques 
but these are not suitable for our mathematics students. The multiple choice could 
be autocorrected but this is not usable for in depth questioning. 

And agmn; 

There is a separate authoring package for creating tests. It provides 5 or 6 
different versions of quizzes and multiple choice questions: list matching. 
Boolean, upload text. Quizzes can also be created directly using an online 
development tool and some HTML. Assessment seems to be seen as consisting of 
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just quizzes. I personally use quizzes only as comprehension tests to make the user 
pause as they click through the material. These are not really assessment tools. 

Granada Leaamwise is produced by Granada Learning in the United Kingdom and 
has been selected to form the core of the National Learning 
Network CAT (Content Access Tool) for FE Colleges in the UK. This tool will 
provide hosting for commissioned materials and will allow all UK Colleges to get 
hands on experience of the content in a Virtual Learning Environment to help in 
their appreciation of and planning for the move to e-leaming. 

LeamWise is a Server system that enables the delivery of Web based e-leaming 
content together with collaboration tools, testing and assessment and tracking and 
reporting. The product is designed for the UK Higher and Further Education 
markets and is currently in use in over 30 FE Colleges. 

LeamWise uses the IMS Specifications for Content Packaging and Enterprise 
Interfaces. 

Granada Leamwise is described as being veiy easy to use for course creation, to be 
well stmctured and to be IMS/SCORM compliant. It is described as excellent for 
student questioning and because it is standards based it can cope with content from 
other suppliers. 

Ascot Systems Course Master and NetTutor is a United Kingdom system that 
allows trainers to rent virtual training facilities for some GB£8.30 per student day 
and deliver training directly to the desktop anywhere in the world. 

Ascot Systems has opened a new Virtual Training Centre offering virtual live 
training rooms - the equivalent of the serviced classroom. Clients can rent and 
operate multiple classrooms simultaneously. Classrooms can operate in tandem or 
in isolation fi-om each other. 



Clients can access their material fi'om Ascot Systems’ server, delivering to anyone, 
anywhere in the world, if necessary utilising existing traditional learning resources. 
The Ascot Systems product was considered excellent for course creation and after 
initial training worked well for course developers. 



ITCampus is compatible with generic software packages (especially 
Microsoft products) and the transferring of course materids/handouts e.g. Word 
documents, html files etc., which were already available in electronic form, was 
straightforward. The other aspects of the course e.g. announcements, user 
instmctions, discussion boards, tests, had to be re-t 5 q)ed into the new platform. 



The original materials were opened in their native programmes, (mostly 
Microsoft Office 2000) and then saved as html versions into the original 
folders. Both material versions were uploaded to ITCampus and the 
performance of each was assessed. In all cases the html versions loaded and ran 
faster than their native versions. The videos for this unit were uploaded and 
checked for loading and running times. The ability of ITCampus to stream video 
from the server gave faster loading times with no dropped frames or stoppages 
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due to buffer underruns. The ease of including or attaching audio notes to all 
materials could be beneficial to students with visual impairments. 

VISIT is described as a Course Management System developed by Eurosync. The 
use of this LMS for course content was described thus: 

Our courses are developed on the StreamMaker platform and uploaded to our 
media servers. They can then be accessed through the LMS - the courses 
themselves are not built through the LMS. We have found the interface easy to use 
and intuitive from both a teacher and developer perspective. All our courses are 
fully interactive, multimedia-orientated This is fully supported by our IMS. 
Questions (both pre and post test) are included as part of the courseware. The 
results of all exams and assignments are reflected in the IMS. 

Student support services 

Student Support Services is that half of a distance education system which 
comprises all the activities of the course from enrolment to examination apart from 
the course learning materials. The provision of satisfactory student support 
services was one of the distinguishing facets of distance systems which 
distinguished them from Teach Yourself Packages and justified the awarding of 
nationally and internationally recognised qualifications for study at a distance. 

WebCT is stated to offer 'plenty of opportunities' for student interaction, but this 
depends on course design. TopClass is said to offer 'quite comprehensive' 
possibilities. 

WebCT, is said to offer both synchronous and asynchronous communication, but 
TopClass only asynchronous in the version used. 

WebCT is said to provide the following student support tools; Quizzes, 
communication tools, external links, a take notes feature and a search tool allow 
students to interact with the course in a number of ways. Calendar, mail, 
discussions and chat (whiteboard) tools can be used to promote student 
interaction. All communication tools can be targeted to individuals and/or classes 
and facilitate communication between staff and students and among student 
groups. A curriculum \ool can be used to embed reading lists. Also dedicated links 
to external reading lists (held at department or library level) can be included on 
the main course menu. General text box or attachment feedback can be provided 
The release of scores and feedback is controlled by the course instructor. 

Another respondent describes the student support tools of WebCT thus; 

1 Shared presentations; student homepages; whiteboard. E-mail; 
Discussions; Chat: synchronous and asynchronous. All these 

COMMUNICATIONS WORK AT COURSE LEVEL AND INVOLVE COMMUNICATION 
WITH THE TUTOR AS PART OF THEIR MANAGEMENT. YOU CAN SET UP 
BIBLIOGRAPHIES/ REFERENCES ETC IN THE COURSE. YOU CAN ALSO HYPERLINK 
TO ELECTRONIC RESOURCES AND LIBRARY RESOURCES. THERE IS A GLOSSARY 

FUNCTION. The system allows mark and textual feedback through 

TESTING AND ASSIGNMENTS FUNCTIONS BUT ALSO ALLOWS RELEASE OF ANY 
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STUDENT MARKSATEXTUAL ASSESSMENTS KEPT IN ‘GRADEBOOK’ FUNCTION. 

Students can see all released feedback, and also statistics on their 

PERFORMANCE IN RELATION TO OTHERS. 



The WebCT tools are available 24 hours a day but the system is manned only 
during working hours. TopClass is described as 'perfectly adequate' and is 
described thus: 



One to one communication through messages in an email system which can be 
tracked is the provision. Plus discussion forums. How you use it pedagogically is 
the challenge but it is a good facility. A whiteboard might be a nice addition. 

WebCT provides a special tool for resources and one can insert links in WebCT, 
and TopClass. 



In WebCT assignment feedback depends on the tutors and their comments are 
easily accessed. In TopClass autocorrection gives results to the students 
immediately but with this proviso: 



The basis is the one to one asynchronous facilities - the server corrects the 
questions and gives feedback and then triggers additional features. If you want to 
work with it you could provide context sensitive feedback but it would take months 
of time to set it up. 



Blackboard's student support tools are listed as quizzes, digital drop-box and 
whiteboard. They are described as very good with chatroom and discussion board. 
Both synchronous and asynchronous communication are supported. It is up to the 
individual tutor to create links to a library. Feedback on work and assignments is 
described as excellent, particularly usage statistics. 



Granada Leamwise is said to support peer to peer and peer to tutor 
communication in both synchronous and asynchronous modes. 



Intranets student support tools are described as: 



Online forums for each module, plus general chat forums, plus forums for dealing 
specifically with administrative and policy issues. Students have read/write 
access. Threaded text discussion. Learners use it regularly. They also have email 
addresses for other students for offline contact, and this also happens. We only 
use asynchronous communication, though the VLE would support synchronous 
chat. There has been no demand for this. Students can email tutors and 
administrators or post questions to online forums. These are responded to during 
office hours. Many readings are posted on the system. To get books on loan from 
the library, they directly connect to the university library, which will mail out 
books to them. There is no direct link between the VLE and the university library. 

VISIT by Eurosync has these features: 



We have a tool called Web4M which provides synchronous communication. Web-4M is a 
comprehensive collaboration and learning environment. It is an integrated suite of peer- 
to-peer, multi-user and groupware tools that delivers information "Just-In-Time ". One 
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of the major advantages ofWeb4M, in any communications or e-Leaming environment, 
is the synchronous environment that it provides. Student to student is supported. 
Web4M^ extends the chat room concept to provide users with a full suite of multimedia 
tools. In each room, users can use audio conference, chat, share graphics via a white- 
board and view slides through an interactive slide show. Web4M can be used as a virtual 
classroom whereby a ‘teacher’ can present slideshows, PowerPoint presentations, or any 
other multimedia format that can be presented over the WWW. Multiple ‘students ’ who 
have logged into this active session can then view this multimedia presentation in real 
time — and participate in a true synchronous environment. It is also possible for the 
‘students’ to see their ‘teacher’. This is achieved using video where a live video feed is 
transmitted to the ‘students ’from a ‘teacher ’s ’ machine. This can be accessed 24 hours a 
day. Feedback on work and assignments is up to the assigned tutor. Technically, this is 
easy and is again achieved by using our synchronous/asynchronous tool, Web4M. 

Tutor support tools 

Another feature of the student support systems in distance learning, which 
distinguished distance systems from Teach- Your self Books and justified the 
awarding of university degrees and other certification for courses done at a 
distance, was the provision of tutor support. Thus in e-Leaming systems the 
provision of tutor support tools is an important dimension which distinguishes e- 
Leaming courses from self study packages. 

In this section we present the evaluation of tutor support tools given by the 
respondents on the LMSs they had purchased or developed. 

WebCT is variously described as 'good', 'perfectly usable' and 'students can be 
tracked easily: 

Students can be tracked easily by groups or course, but not individually across all 
courses. Tutor can view areas visited and tools used 



WebCT's provision of tutor support tools is further described: 



Full tracking of student activity and performance is available from a simple 
MANAGE STUDENTS menu. Group discussions and assignments can be 
facilitated using the student presentations tool. WebCT 3.x campus edition quiz 
authoring tool is not the most easy to use. However, the Respondus tool (which 
integrates with WebCT) is very easy to use in this respect. Student progress etc. is 
a function of the manage students tool. We have fully integrated WebCT with the 
course student and staff databases using the IMS enterprise API tool. 

And again: 

In WebCT we can export mid and end-year grades to MIS. There is the usual 
array of communications tools, plus gradebook for progress, plus tracking of 
every student visit to course pages. For the preparation of questions and 
assignments much depends on the tutor ’s IT skills: many start with Word and 
move on to more multi-media and interactive methods, but it takes time. 

Tutor support tools are said to be a good feature of TopClass: 
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This is a good feature. The data is easily accessible and you can collect different 
sets of information according to your need. The facilities are good and you can 
create classes or courses and easily modify them. 

It is a good system once you have figured out how it works. For the list of students 
you get: mail/look student's profile/coursework/review of submissions they have 
sent in/test progress/you can look at what pages they looked at/ when they did 
so/how much they looked at/ what Yo of material each student looked at/what % 
each student has clicked on to/which actual pages each student has clicked on. A 
brilliant facility. 



Groups can be allocated in WebCT. The TopClass facilities are described as 'good' 
and described as: 



There is a create/edit class facility. On day 1 you associate students and 
instructors for each course. There is batch registration of students: you can 
automatically register 200 students in 1 minute providing their full names in a 
pre-ordained format, the server then, according to set guidelines, produces 
usernames and passwords. A very useful facility 



On the question of the preparation of questions and assignments by tutors WebCT 
is described as 'the LMS is fine but tutors need training', 'given training the system 
for design of quizzes/tests is simple' and 'moderately successful'. TopClass has an 
authoring package for creating multiple choice questions but developing questions 
is time consuming. It is described: 

Limited facility. Creating question pools is lengthy and awkward. This is added to 
the limited nature of the question types in the system. 



Monitoring and planning by tutors for student progression in WebCT: 



Tutors can see which pages have been viewed by which students, how many 
discussion messages students have read, how often students have logged on 



In TopClass there is a flexible monitoring system: 



There are already a large number of monitoring tools for what students studied 
and submitted. You can get the average mark of the class on assessments. Class 
progress can be easily studied. You can assign a time frame to parts of the course 
or the whole course. You can make Part 2 available only when successfully passed 
Part 1. 



There is said to be no provision for tutor to institution communication in WebCT. 
TopClass 5 is said to have an excellent system: 

In general tutors have too few tools and administrators have too many. Changing 
the defaults is tricky. Top Class 5 has a campus wide administration system which 
works with an Oracle database. You need a specialist for Oracle. 

Tutor support tools in Blackboard are said to be easy to use, group management is 
available, there are facilities for the construction of simple questions (multiple 
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choice) and online marking of tests (quizzes). Another respondent stated that the 
trac kin g of Students was easy and efficient and that there were excellent facilities 
for group management and preparation of questions. A further respondent stated 
that the tutor support tools were very userfriendly as was the preparation of 
questions. For monitoring student progress there was the grade book, course 
statistics and the number of specific hits. 

For Granada Leamwise the present version 1.2 is said to have good reporting 
facilities which will be improved in version 2. The linking of the LMS to other 
institution recording systems is given thus: 

As this is an interoperable product (IMS and SCORM compliant) interoperability 
between the VLE (LMS) and Student Record Systems (SRS) is ‘relatively ’ easy for 
all commonly used systems in the sector. Between Capita systems and the VLE 
these are highly developed for interactive communication. 

For FDL Learning Environment 2.09 it is said that the student management system 
is relatively straight forward, and reports can be generated fi'om the student 
database but there is no link at present between the LMS and the MIS. 

VISIT by Eurosync lists the facilities that are provided by the LMS to the tutors 
for managing their group(s) of students as: Adding, editing and uploading course 
material; maintenance of reading list for both online and offline content; 
correction of tests or test sections which cannot be corrected automatically by the 
system, scheduling of classes. The system provides tutors with access to the 
following administrative tools for monitoring and planning student progress: 
responding to particular questions on forums and moderation of forums; online 
mentoring; LMS provides up to date statistics on all aspects of student progress. 
It is possible to drill down to course level and generate reports on individual 
students or groups of Students. 

Administrative systems 

This section deals with the provision by the LMS for student enrolment and 
collection of fees; for the provision of passwords and system security; for the 
provision of a student records database; for the collection of student data for 
course results; and for the provision of administration for courses, classes and 
tutors. 

Paulsen(2002) has recently shown that LMS administrative systems contain various 
subsections: course creation tools, learning management systems, student 
management systems and accountancy and has argued that ideally all these systems 
should be integrated together. He has proposed a Jig-saw model of online 
administrative systems and a Hub model. He writes as follows: 

The Jigsaw model is a simplistic model used in the web-edu project. It includes 
the four main categories of online education systems that are listed below and 
presented in Figure L. 

• Content Creation Tools (CCT) 

• Learning Management System (LMS) 
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• Student Management System (SMS) 

• Accounting System (AS) 

It is called the Jigsaw model to indicate that these systems should fit together to 
exchange data more or less seamlessly. The figure also presents some examples of 
actual systems and shows how the IMS specifications relate to the systems. 




Figure 1. The Jigsaw model for online education systems (Paulsen) 

The question on enrolment and fee registration systems was not clearly answered 
or perhaps misunderstood. WebCT is variously described as having 'no enrolment 
system', 'none in this version', 'not aware of system' and yet 'enrolment system is 
interoperable with our main system'. Again it is stated that enrolment is done by the 
university registry using the existing student record system. 

TopClass comes out well here; 



There are two enrolment systems: the Batch System which assigns passwords to 
students as they enrol and the Apply to Enrol and be enrolled System, both of 
which work well. 



Most of the other systems do not have online registration, but VISIT by Eurosync's 
LMS 'uses a checkout system, EasyClear. New students would fill out a 
registration form, get a username and password and they would then be able to buy 
courses online.' 



On security WebCT is described as 'as good as any http system', 'username and 
password for access', 'satisfactory' and 'very good'. TopClass too applies user name 
and password access and is considered fairly watertight, on condition, one 
respondent adds, that students do not give out their passwords. 

No problems with data security were reported by any of the respondents, with 
Blackboard being described as 'good' or 'no problems'. 

VISIT by Eurosync has various levels of security and password control; 

Public users have access to the site and all elements not protected by the 
user management and security. 
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Registering Students are required to have a login account to access the 
element of the site based on their permissions. For example access to 
interactive courses and tests. They will have access to their test results 
and their progress on their selected courses. 

Access to schedule showing online classes, registering for classes. 

Each student has access to information showing their progress on various 
aspects of their course. This would cover for example their exam results, 
progress on their projects and viewing their reading list. 

Student record database in WebCT is variously described as 'limited by flat format 
but will move to Oracle in 2002', 'adequate', 'seems fine' and 'very successful and 
stable and easy to manipulate'. Some integration of systems is indicated by: 



We can link it to our IMS and download data. The API is advanced in 
the VLE world and conforms to IMS. Data is stored at course level: 
i.e. I can easily get data into courses and the global database; it is 
not as easy to get information from courses, though you can export 
into global spreadsheets. 



The student record database is described as 'one of the great strengths of the 
TopClass system'. 

VISIT by Eurosync uses an active/passive SQL cluster on a Windows 2000 
platform which is both efficient and fast. 



The question 'What structures are provided for recording data for 
certification?' brought for WebCT the answers 'none at the moment', 
'grades are recorded' and 'will be integrated to our central system', 
'we would not do this. We use our MIS as the single data source. 
Most Examining Boards will not accept electronic source evidence'. 



In TopClass: 

All the discussion forum is archived. To check class grades you go into the student 
area. When did they do the tests? Haw many times did they sit the tests? This is a 
pretty good feature. Subject always to the quiz mentality of the system. 

'What facilities are provided for administration of courses, classes and tutors?' 
brought the answer for WebCT that there is a global database and individual 
course databases and that it can manage courses, students and files. 

TopClass has an extremely flexible system which is described as 'fairly simple to 
use and good for assigning tutors to groups': 

Extremely flexible system: perhaps there are too many choices with different 
privileges at different levels. One problem worth mentioning is that to find an 
individual user you have to search all users rather than have a search facility for 
individual classes. Also: Course material is not the same as class material: which 
can be confusing. 
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VISIT by Eurosync reports: 



The facilities provided for the administration: secure access for appointed 
staff that have access to control all aspects of the web system. The main 
sections are: 

• Teacher/ Tutor administration; Adding, removing Teachers from the 
system, Setting permissions for Tutors 

• Student administration. Adding removing Tutors from the system 

• General Site Management, Adding, editing sections 

• Content Management, Approval of content added by tutors 

• Forum Administration, Full control over all forums, setting topics and 
access for all users. Responding to feedback from all lower lever users 

Another level covers the features of the system tutors will have access to: 
Adding, editing and uploading course material. Maintenance of reading list 
for both online and offline content. Correction of tests or test sections 
which cannot be corrected automatically by the system. Scheduling of 
classes. Responding to particular questions on forums and moderation of 
forums. 



Technology 

Technology has been used widely in educational systems since the 1930s when 
Plessey and then Skinner set out to design 'the perfect teaching machine'. A major 
distinction between face-to-face and distance education systems is that in face-to- 
face education the technology is a supplement to the teacher, while in distance 
education it is a substitute for the teacher. In electronic education or e-Leaming 
the role of the technology becomes crucial for the operation and success of the 
system. 



WebCT is described as: 

The software has been robust and is subject to continual development and 
improvement Servers depend on the institution (locally installed rather than 
hosted). WebCT lives on the server: only client issues are browser settings, 
which often need altering from normal defaults. To my knowledge there is 
no metatagging, though the next versions of WebCT allow resource sharing 
of learning objects at supra course level, so I think the repository and 
metatagging is to come. As WebCT is a leading member of IMS, they are 
absolutely committed to this development The Vista range (next generation) 
is aimed at interoperability and allowing third-party software to interface. 
We are not large enough to stretch it I believe large user databases have 
60,000 students. There is the facility for load balancing across servers and 
multi-institutional use. Speed is OK on our network, but this has been 
upgraded We have not used a lot of multimedia materials, so cannot say. We 
tend to advise tutors to keep file-sizes small and think of 56k modems. 

WebCT runs on a range of hardware and software platforms and was described as 
'acceptable and well integrated'; one institution used it as a standalone system 
running with an Apache server. 
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For TopClass; 



All you need is an NT server or a Windows 2000 server and students don't need 
any software - a big advantage. It runs on UNIX and Mac as well. 

WebCT runs with all browsers though bandwidth is said to be a problem. The 
current version does not permit metatagging but the next version will. TopClass is 
described as; 

Every page of Top Class is in HTML. It can be used with any browser. The .plug 
file format will only work with Top Class. It is very good at tracking pages or 
courses or tagging to help you to find pages: useful when pages get dislocated. 

The Intranets website is hosted by Intranets in the US. 'As our system is all text- 
based we have not had serious difficulty as regards access times. The site is not 
integrated with our office systems'. 

WebCT is described as 'perfectly flexible and frequently updated'. It is described: 

Flexible but also limited by the designers level of creativity and their pedagogical 
expertise. It is possible to integrate other software within the software structure 
a^ we went with an internationally recognised, well established software 
provider to make sure there would be regular upgrades of the technology. They 
were particularly supportive of academic’s needs as a lot of their work is to 
support this market. 

Top Class is 'relatively flexible but requires a knowledge of how HTML handles 
graphics' and: 

The didactic structure is not really flexible but you can use it cleverly. It could be 
worse. Because the pages are web-based one can integrate it with other software 
or systems. 'Click here' to launch NetMeeting is a possibility. One could stick in a 
chat system. Top Class 5 is said to work brilliantly with new systems. 

One institution states it has created over 7.000 courses on WebCT and enrolled 
aver 24.000 students and the system coped without a problem. 

Another institution had no trouble with 8.000 students on Web CT and it was 
described as very good for large numbers of students, courses, tutors. There were 
no difficulties with TopClass 3 but differences between TopClass 3 and 5; 

There are differences between Version 3 and Version 5. Top Class 3 you bought 
on the number of enrolments at the one time. Top Class 5 you buy by the number 
of registered students in the system. Oracle lets Top Class 5 use a minimum of 
1000 students. The basic Top Class 3 licence was for 30 simultaneous users. We 
have had up to 1000 students on it without a problem. 

Blackboard Level 1 is said to accept up to 10.000 enrolments; Blackboard 
Learning System more than 10.000. 

Granada Leamwise accepts metatagging. 
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WebCT was described as being limited only by client bandwidth with one user 
stating they decided to use only low bandwidth content because of end user issues. 
TopClass was running on a normal desktop not a specialised server. 

I 

VISIT by Eurosync is analysed as: 



The courses will run on Internet Explorer 4 and above on any pentium machine. 
StreamMaker allows you to stream the courses on nothing more than a 28.8k 
modem with only a srriall amount of latency. The system does permit metatagging. 
The didactic structure is flexible and is determined by the developers who design 
the course rather than being determined by the underlying technology. The 
technology can be upgraded if new updates become available. The capacity of the 
IMS is dependent on the capacity of the web server farm that hosts the site, 
database etc if the course is being done online. Almost all our courses are 
intended for online use. In theory it could handle thousands of concurrent online 
users as long as there was enough capacity on the web servers. StreamMaker™ is 
the basis for all of UniqueLeaming.com' s streaming eleaming solutions. 
StreamMaker™ uniquely enables the creation of full screen, fully synchronised 
interactive multimedia, delivering graphics, animation, text and audio over the 
Internet and corporate Intranets. 



Price 



Investment in an eLMS for e-Leaming is not a cheap decision. In recent years there 
have been considerable price increases for the purchase of and use of LMSs. There 
have been claims that some producers have abandoned the academic market of 
colleges and universities in favour of the corporate market. These price changes 
have led some institutions to change suppliers, or even to consider the possibility 
of developing their own LMS. 



The price of WebCT is given as $5,000 per year but this is qualified by another 
respondent who states that an unlimited standard edition licence is now in 2002 
$7,000, up fi-om $5,000 last year. A further respondent quotes $12 per user and 
another says that the institution negotiated the price. Another system manager left 
these questions blank which may indicate that the price was not known to the 
system manager. 



The full cost of WebCT for a year is given as: 

2 This year’s unlimited user license for WebCT Campus is $29,500. 
Maintenance costs are difficult to say. I work fairly full-time on it, 

BUT TRAIN, ADMINISTER AND SUPPORT AS PART OF THIS, AS WELL AS 
DEVELOPING SOME MATERIALS. WE DEVOLVE E-LEARNING WITHIN THE SYSTEM 
TO TUTORS WITHIN OUR SCHOOLS. MUCH OF THIS TRAINING OF TEACHERS AND 
LEARNERS AND SYSTEM USERS IS ABSORBED, BUT WE ALLOW THE EQUIVALENT 
OF020.000(€3 1.250) PA. 



Blackboard Level 1 is given as £4.500 (€7.031) pa and Blackboard Learning 
System as £22.000 (€34.375) pa. In addition maintenance costs are said to be 
massive: a Blackboard administrator, Blackboard trainers, IT technician support; 
^ 
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generation of content costs more than the LMS. Another Blackboard respondent, 
however, gives costs as $295 per course per annum. 

The cost of Granada Leamwise is not known. However, the respondent remarks: 

The cost to the institution is much bigger than the cost of buying the vle(LMS). 
Huge development costs are required to get the culture change necessary to take 
on such technology. This is not unique to this system it is true for all. 
Maintenance costs: VLE Administrator (Online Systems Engineer) is the only one 
dedicated to the VLE itself. However a specialist content team and mentoring 
team are required as well as management. Training of teachers and learners and 
system users costs are enormous. 

The price of T opClass is described dramatically as : 

Top Class 3 was £1.735 including VAT, but this may have been an institutional 
price, for 30 simultaneous users per year - this means that 30 could log on 
simultaneously. Top Class 5 allows a minimum of 1000 users at £18 per head per 
year for a total of £18.000 per year for a 3 year contract. Then you have 
maintenance fees plus an Oracle database plus staffing. We consider that Top 
Class have turned their backs on the higher education market and have decided to 
change to WebCT. 

Another respondent states that the cost of updating is prohibitive so 'we have 
decided to design our own LMS'. 

The WebCT st affin g is given as one academic per year and half a technician; 
administration and help desk support; none - hosted by supplier; need financial 
support for staff training and purchase of servers. 

One institution had a team of 8 catering for 2.100 students online on TopClass, 
another stated: 

There are two answers to this. There was no problem with Top Class 3. Until 
recently no problems since installation by an IT specialist which took half day. 

Top Class 5 is horrendous. It took one full day for a person from WBT Systems 
plus 2 Oracle specialists and there have been many problems since. 

The costs of Ascot Learning System Course Master is put at £40.000 ( €62.500) 
per year plus one IT technician and 100 hours of staff training time. 

The FDL Learning Environment 2.09 is currently free as it is a pilot institution but 
it is said that training materials are poor and the college has to provide technical 
and pedagogic training materials. 

Intranets costs $150 per month with no maintenance or training costs. 

The costs of IT Campus are not known but the college allows £15.000 (€23.437) 
per year for maintenance. 

BEST COPY AVAILABLE 
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VISIT by Eurosync costs €40.000 to buy with an annual fee of €20.000. 
Maintenance is the responsibility of the IT manager and training of the teachers and 
learners is not a concern. 



On the training of teachers and learners and system users there was a range of 
responses: for WebCT - 35 to 40 days of online training per year; reseller to train 
staff at £500 per day; 18 hours per lecturer; provided centrally from departmental 
funds. For TopClass the institution had to develop courses for both staff and 
students. 



Evaluations 



Respondents were also asked to give their overall assessment of their satisfaction 
with the LMS they had purchased or developed and what features they would like 
to see included in the LMS in the future. Although these questions were additional 
to the questionnaire, most respondents took the opportunity to add further 
comments. 



Of the WebCT users who replied to this question one stated 'very good and 
userfriendly' the other 'ideally suited, efficient and adaptable'. Another states 'We 
have found WebCT to be an excellent tool for implementing e-leaming across the 
institution'. Yet another Excellent for us, although we recognise it is not perfect'. 



On Blackboard 'Satisfied - we plan to upgrade from Blackboard Level 1 to 
Blackboard Learning System'. Again Elackboard was very simple and easy to use - 
would recommend it'. 



The view on Intranets gives a different perspective; 

Generally, this system meets our needs. It is simple, unfiissy, and easy to learn. In 
the long run, we would like something more sophisticated, but we found when we 
were looking for a system that providers wanted to give us greater functionality 
than we required and they wanted us to be champions within the university for 
fully integrated campus systems, rather than provide something that would allow 
us to get on with our job. 

There is a touch of sadness in some of the evaluations of TopClass: 



7 have looked at Blackboard and WebCT. I would sum up the Top Class 
experience as initial excitement leading to eventual dissillusionment. We wanted 
to support it because it was Irish and had been developed in Ireland and was an 
Irish world leader. But our contacts with the company were less than helpful. They 
are, unfortunately, more interested in corporate clients. We are going to move to 
WebCT 



And again; 

We were quite satined with Top Class 3. It was a bit non-intuitive for developers 
and there are bugs in the authoring system. It was short sighted of Top Class to 
move from the higher education market. The Top Class authoring system would 
have been good if it worked as it was supposed to. We have decided to move to 
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WehCT as TopClass have left behind the HE user in favour of corporate clients. 
Top Class 5 is also too pricey and runs on Oracle which requires a very 
specialised expertise to configure and maintain. There are few HE institutes still 
using Top Class today. WehCT bills itself as the world's "leading provider of e- 
Leaming solutions for higher education " so hopefully they won 't abandon their 
HE clients - Blackboard then seems to he more for K-12. 

VISIT by Eurosync gets this judgment: 

Our emphasis is on creating and providing courseware to corporate and 
individual clients. We are not IMS specialists but do consider a solid and 
adpatable IMS as a key consideration in our corporate strategy. We find that our 
current IMS suits all our needs at this point in time. The previous IMS we used 
was inherited from Leam2 but we felt it didn 't suit all our needs. 

A further user states 'we are not satisfied with what is on the market and will 
develop our ovra'. 

Finally respondents were asked what changes they would like to see in the future 
to the LMS they had purchased. 

The WebCT users said 'compliance with LMS standards' and 'global tracking of 
students, cross course emailing and EMS/IEEE standards'. Also 'Improvement of 
question and testing formats'. 

On Blackboard the request is for 'Integration with other systems - student records, 
examinations, photos, more security, faster speeds'. And also 'Integration of 
enrolments etc'. 

FDL Learning Environment is asked for 'Continued support for Macintoshes and 
ability to successfully import and export IMS content packages'. On Intranets 'We 
would like some form of student tracking integrated with the system'. On IT 
Campus 'Full enrolment and student record system'. 

For TopClass: 

In the Top Class system I would like to see a much shorter run up time for the 
tutors: something simpler - an easier path from beginners to improvers to 
advanced Many of my colleagues were put off in the beginning. They were 
blocked from having their notes in Word to putting them into the Top Class system 



Top Class would he nice if it was not so templated It needs some synchronous 
communication, including desktop conferencing. They will have to offer easy ways 
to do things like inserting Shockwave and Flash content and streaming audio and 
video and intelligent whiteboarding. It needs to be graphically more inventive 
and user-friendly: in displaying courses Top Class needs to go beyond this 
overly-repetitive system of clicking on a left frame index to view content on the 
right. At the moment TopClass seems to be predominantly made to bring together 
simple "text and gif "-type pages which don 't take advantage of the unique ability 
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of the web to host highly rich media content and support all kinds of collaborative 
activity. TopClass and other LMSs in the near future will need to offer user- 
friendly authoring packages for teachers and trainers to create simulations and 
courses with a virtual reality dimension. 

For LMSs in general: 

In the IMS market in general I would like to see standards implemented more 
rigorously, for example more compliance to standards such as SCORM and 
AICC. Big companies such as Macromedia and Microsoft demonstrated tools 
that will enable easy creation of SCORM-compliant learning objects. It is 
importcmt to keep up with future developments. 



The overview conclusion was one of criticism of TopClass for abandoning the 
higher and further education market and concentrating on the corporate market. 
WebCT and Blackboard seemed to give general satisfaction to the higher and 
further education market. One institution had designed and produced its own LMS, 
and one was planning to develop its own, and they seemed satisfied with this 
solution. 



Conclusions 



The following conclusions can be drawn from this analysis: 

1 . In the United Kingdom and Ireland there is a very extensive implementation 

of e-Leaming via LMSs. This includes provision at degree and diploma 
level. It seems that very many universities and colleges have purchased an 
LMS, and many corporations too. 



2. The overall impression is the domination of the scene by the major 
American-based LMSs, notably WebCT, Blackboard and TopClass. This is 
because of the use of English in the United Kingdom and Ireland. 



3. All institutions are sensitive to SCORM and IMS standards and 
dissemination of these standards has led to them being considered almost as 
a norm. 



4. WebCT has pushed hard to become the market leader with extensive 
promotion and presence at e-Leaming conferences. 

5. There seems to be general satisfaction with WebCT as a user-fnendly, 
competent product. 

6. Blackboard has given general satisfaction but is less widely marketed than 
WebCT. 



7. TopClass is praised for its student and records database. 



8. Recent price increases, often quite considerable, are a feature of the market 
with prices in the range €20.000 to €50.000 being quoted. 
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9. Data produced by the LMS systems are not yet generally integrated into the 
institutions' administrative databases. 

10. The concept of quizzes and multiple choice questioning, a feature of most 
American LMSs, is not considered adequate for European academic 
evaluation. 
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The use of Learning management systems in Germany 

by 



Helmut Fritsch and Holger Fdlimer 



Abstract 



17 systems have been analyzed during 2001 and 2002 in Germany. This number is not representative 
for the increasing use of LMS . 

Most systems still are under development. The origin of LMSs in Germany coincides with a structural 
reform of university administrations. The development of LMSs has been pushed forward by faculty 
members of computer science on the one hand and administrations on the other. Half of the systems 
interviewed come from universities the other half partly from public further education agencies and 
private business for inhouse training. Most LMS are self-developed, only a minority has been bought 
“off the shelves”. The scope of the LMSs used is restricted in most cases to didactical elements not 
covering the administrative and bookkeeping possibilities of professional systems. Even in some 
cases, where bought systems are used some useful elements are deliberately cut off. Some of the 
self-developed systems follow the “open-source-policy” thus hoping to interest more users and 
decreasing cost of universities at large. Almost 70.000 learners are covered by the systems described. 
The lack of content for the LMSs is deplored in several instances. Meta-tagging and standardization 
issues do not yet play a prominent role. 



Country 


Language 


Geographical 
Area (sq.km.) 


Population 

(millions) 










Germany 


German 


248,577 sq km. 


82 259 500 



A. introduction 



The definition of LMS as presented by Desmond Keegan in this book does not meet 
the situation of LMSs in Germany. Most systems still are in development. 

We found three forms of LMS systems in Germany: 

• 1. The all-inclusive-system: Large portions of curriculum material already exist 
in some digital form and the promise is to offer all necessary elements via 
Internet, including enrolment, invoicing and administration. 

• 2. The second form is course-offer in a section of an institution (and the goal 
to persuade the rest of the institution to all do it alike), the courses are special 
and relevant for only parts of the institution (like the "sub-insitution" 
department for further education) 

• 3. The third form seems to be the specialized alternative where only parts of a 
curriculum are meant to be put into an LMS, parallel to existing other, mostly 
face-to-face forms. 



There are some factors of complication which we found in researching projects which 
use learning management systems. These factors first have to be isolated in order to 
come to a set of data which is comparable for interpretation. Lets take as an example 
the interview with one of the most known systems in Germany with more than half of 
a university's population to work with it: the LVU system of FernUniversitat. Emerging 
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from three parallel initiatives in three different sections of the university the first steps 
have been made to integrate the various aspects of a true LMS into one computer 
based "system". It still runs as a project, the third version will emerge in 2003. There 
are some courses already functioning totally via the web. 

If we take a closer look at this special system we may find that it emerged from 
academia, faculty members from different fields cooperated and have been merged 
into one big project. But: Parallel to this effort there is the classical university 
administration system with a much longer history than the lifetime of this university 
could tell - so called kameralistic procedures of bookkeeping and administration, not 
having changed for almost a century run parallel to such "modern" efforts to build a 
completely integrated system. Indeed as long as the university is state-funded and 
responsible towards the ministry of education and science, the yardsticks of this 
organizational structures will be kept in practice. This means that parallel to the 
project effort of LVU there are other state-wide projects to instal reporting systems, 
building of indices for efficiency and throughput of products (student qualifications), 
reforms of bookkeeping and a whole set of separate projects to trace the cost flows 
and workforce data within a university. 

For the last 15 years now the theme of quality management in public services has 
been a major topic. TQM measures and literature and projects in Europe have 
influenced thinking of what public services are about. We came across these 
initiatives in the framework of community politics and the so called Tilburg model - 
meanwhile there is a global perspective on community reform in that focus more and 
more is laid on the services aspect and the citizen as a customer than ever before. 
New public management is one of the outstanding catchwords in this filed. So it is no 
wonder that also university administrations try to cope with these new demands. It 
certainly has been kind of a revolutionary act of the former chancellor of our 
university, (the chief of administration), to fomulate the "goal " of university 
administration: "to make research and teaching possible". 

Influenced by the didactical definition of distance education and calling the institution 
that offers it "helping agency", a term coined by R. Manfred Del ling in the early 
seventies, this view of the new role of administration : "facilitator" of research and 
teaching makes it understandable that also university administration today functions 
according to some principles of new public management. And such an administration 
will also strive for cost reduction and increase of efficiency. And create models for 
management decisions instead of just keeping to a set of standardized behaviour. 
Modern university administrations are not any longer comparable to laboratory rats 
trained to always behave within a fixed set of conditioned reflexes. 



B. Double dichotomy 

Whereas we were used to make a structural and analytical difference between the 
public education sector and proprietary education in Europe (mostly on the level of 
vocational education ), there seems to have emerged a new dichotomy that plays a 
major role in the development of web education systems. The dichotomy between 
academia and administration. ; 

If you run a university according to a model of private enterprise you are likely to 
have an administration that is not inclined to research and develop a complicated 
system of learning management. The system most likely will already have been 
developed, tested and sold elsewhere and there will be a "community" of users of 
such a system becoming a more and more powerful partner for future developments. 
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Only very large cx>mpanies will be inclined to have a distrust towards such systems 
"off the shelves" and only if they have own resources for the development of such 
rather complicated computer programs. Some of these will develop such systems in 
the hope of gaining refunds when the system is ready for the "market". These 
systems will have incorporated all known elements of LMS in order to be able to 
compete in the market. This is because their administration is not separated from the 
enterprise but an integral part of the business. 

Not so with German universities: There is a tradition that a bright computing center 
does not need to buy programs developed by others because the need to buy 
external programs would question the qualification of existing personel in such 
centers. 

Evenmoreso when an academic department for computing science is involved: staff 
and students would always resent to accept external programs because of the need 
to prove the everyday quality of own programs as a symbol for the institutions high 
ranking capacity to be forefront in computer development. 

The task of such a computing center indeed will necessarily produce a conflict 
between the university's administration which hoped for increase of effciciency and 
decrease of performance cost and the department which always has to look for 
proofs of their respectability tested in - at least- the "own" market. This seems to be 
the reason why so many university based IMS's in our study refrained from buying 
off the shelves. The logical consequence in higher education development with state- 
funded universities is, that as soon as universities administrations find out that 
revenue from such LMSs is less than promised they must go for "own" developments 
to create LMSs or like systems which incorporate the systemic features of such 
systems also for departments not so inclined to prove their self-confidence by 
promoting "own" systems. They then will believe in the capacity of larger 
administration oriented computing systems at least in the dimensions of SMS 
(Student Management Systems) and AS (Accounting Systems). 

That means we always will find the typical dichotomy between academia and 
administration. Hopefully both sides will in due course learn from each other. 



C. The results of the interviews in the web-edu project ^ 

The projects interviewed in Germany alone represent som 70.000 learners who 
already have experience with learning management systems(LMS) of different kinds. 

This number can by no means called actual (the interviews have been carried 
through in the year 2001 and 2002) nor representative. 

Yet we do hope to show some trends steming from the projects we encountered, 
which were on the one hand open for such an interview (we met quite a number of 
actors in this field who refused to have an interview done, sometimes even with harsh 
comments) so here is the place to thank all those members of staff who had been 
willing to do the interview and thus contributed to this Leonardo project! 

Categories of comparison 

Indeed, it is a very big difference to interview an assistant on a part time basis from a 
chair of a technical university who does a good job in finding a solution for some of 
the many tasks of teaching, thus helping the professor to concentrate on other things 
compared with the professional department of human resources of a multinational 
enterprise using all the features of software they have developed an used for years 
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now with a fixed cleintele of in house trainees suming up to 20.000 people. 

As early as 1997 ZIFF studied the features and compared performances in the field 
of LMS. 

The following remarks have to be read with some precautions in mind: 

a) the types of institutions are various - we find universities who decided as a vi^ole 
to set up an LMS and even consortia of universities. 

b) Then we find smaller units from universities who decided to use an LMS and even 
single chairs to use something like that - self-developed and functioning ; we 
described this as "small but beautiful". 

c) On the other side we find companies, who try to get hold of a portion of the 
expanding market by presenting learning-units and invoicing their clients while 
they learn. 

D. Overview of the German projects interviewed 

The following table is already a shortcut version of the data found in the interviews, 
The size of the data collection made it necessary to use abbreviations and to 
condense the information. We think however, that this process of focussing is a 
legitimate procedure to come to an overall interpretation. You will find in the following 
tables always a fixed set of data about the project in the beginning of the table. We 
have been thinking about constructing a huge database with all the information of all 
categories included: This would have been too large for any screen or paper- which 
is the reason for splitting up the answers into the sections of the questionnaire. In 
doing so we found - again - some items omitted or not paid due attention to them, like 
the questions of metatagging, accessibility or the "open source policy" of many of the 
German projects. 
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Lis t of German projects analysed 



Institution Identification 


LMS 


ff students 


# courses 


# tutors 


Virtus. Cologne 


University 


ILIAS (SD) 


12 000 


50 


30 


Teleakademie Furtwangen Tech. University 


Black- board BSCW 


500 


10 


50 


Netzentwurf. Karlsruhe 


University 


SD 


100 


4 


5 


Virtuelle FHS Luebeck 


Tech. University 


Black- board 


1000 


130 


40 


LVUHaeen - 


University 


VU(SD) 


27 000 


270 


100 


media-design. Munchen 


private 


Leamlink evoeye 


700 


5 


130 


Osnabrueck Virteller Camnus University 


VC Prolog Tutor (SD) 


80 


3 


8 


Virtuelle Universitat Regensburg University 


planetux (SD) 


280 


57 


35 


Darmstadt CSGDl 


private 


wave learn (SD) 


800 


2 


10 


ru Chemnitz. VDE 


Techn./ University 


(SD) 


1000 


5 


20 


akademie.de asD GmbH 


private 


(SD) 


1700 


60 


none 


IPTS Kronshagen 


public 


interwise- ecp 


1000 


350 


130 


SAP AG 


private 


Learning solution 


20000 


500 


10 


Akademie Uberlingen 


private 


lemen -im-netz (SD) 


112 


24 


5 


Virtuelle Hochschule Oberrhein , University 


clix campus 


200 


b 


n.a. 


ZWH Diisseldorf 


public 


DLS jfrom ETS 


n.a. 


39 systems 


>110 


BildunePlus eLearning 


private 


corporate learning 


n.a. 


25 


40 




E. Detailed Analysis according to the sections of the questionnaire 
Methodology 

The following sections, as indicated above, represent the answers to the 
questionnaire which we agreed upon in the project. We would suggest to keep in 
mind that the abbreviations used in this table do not necessarily represent the exact 
wording presented by interviewees but we tried to boil down the meaning of the 
answers into one or two expressions which in turn try to make the results 
comparable. This methodology arises with large data but is a new form of research in 
that after a while it will be common to present a text with hyperlinks back to the 
original wording. Imagine that there is a huge database with all interviews from all 
partners and answers to every single question: it easily surmounts the intellectual 
capacity to present something and even draw conclusions which have to be sound 
and must be ready to be proved by anyone ready to analyse the whole corpus of 
interviews himself. This technique can easily be presented in the WWW, where via 
hyperlink in each cell you travel to the original item. 
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1. Course development tools 



Institution I 


LMS 


i 

studs 


cour 

ses 


tutors 


l.lCourse i 

creation i 


1.2 i 

didactics | 


1.3 Use 


1.4 media 


1.5 1 

assessment 1 


Virtus, 1 

Cologne 1 


SD 


12 000 1 


50 


30 


possible 


open 1 


easy 


WWW 


no 1 


leleakad. i 


BlackB. 


500 1 


10 


50 


no 1 


open 1 


easy 


WWW 


self 1 


Netzentwurf | 


SD 


100 1 


4 i 


5, i 


no 1 


open 1 


1/2 hr. 


WWW 


yes j 


V^irtFHS 


Black-B. 


1000 i 


130J 


40 


no 1 


flexible 1 


eagf 


20 formats 


yes 1 


LVU Hagen | 


(SD) 1 


27 000 1 


270j 


100 j 


not yet 


flexible | 


easy 


not all 


automated- | 


ri 

1 

? 

1 


Leamlinkj 


700 i 


5 i 


130 1 


satisf. 1 


flexible j 


2 days 


all 


help 1 


Osnabr.VC i 


SD 


80 1 


3 


* 1 


good 1 


exercises | 


in 

developm. 


JAVA 


e-mail 


VURegensb-l 


SD ^ 


280 


57 


35 1 


no j 


open j 


easy 


all 


no I 


SGD j 


SD i 


800 i 


2 


10 1 


no 1 


flexible ! 


not easy 


depends 


mail I 


TUC.VDE j 


SD j 


1000 1 


5, ! 


20 ! 


no 1 


flejdble j 


notea^ | 


all 


perfi^ 1 


asp 1 


SD 


1700 1 


60 


- 1 


not 1 

satis&ctoiy 


not flexible 1 


not easy 


graphics / 
audio 


mails 1 


EFTS 1 


ecp ^ 


1000 1 


350 j 


130.^^ J 


yes j 


flejdble i 


2 days 


all 


online j 


SAP AG 1 


SAP-LS 1 


20000 


5mj 


10 1 


medium | 


flexible i 


2 days | 


all 


online | 


1 Akad.U. 1 


SD 1 


112 


24 J 


5 ■ i 


no i 


fixed i 


simple i 


aU 


self 1 


IvHOberrh. | 


clix 

campus j 


200 j 


6 


n.a. 1 


no i 


depends 


good 


all 


self 1 


ZWH 1 


DLS i 


n.a. i 


39 1 


>110 j 


no j 


flexible 1 


1 day 


all 


plug-in 1 


iBildungplus I 


corp. 


n.a. i 


25 


40 


no- 1 


§ 

flexible | 


user- 

friendly 


all 


yes 1 



1.1 Course creation 

In the majority of cases there is no course creation with or inside the LMS in fact 
independent of the origin of the LMS - self-developed or purchased. Institutions which offer 
specialised courses as supplements for face-to-face lectures, target-group orientated 
courses or just space for online-cooperation use their LMS for creation of exercises too. 



Four institutions declare their LMS as satisfactory for course creation - these LMS are self- 
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developed. One of the big projects rejects to develop courses v^ith the LMS, "because we 
don't want to become dependent on one provider". 

1.2 Structure and didactic flexibility - openness. 

The answers to the question with regard to "structure and didactic flexibility" permit 
the conclusion "anything goes". The answer extend from "complete flexible" to "all 
what a browser can present is possible". Only four institutions work explicit with a 
didactic frame work (which is changeable too).There seems to be no limits of didactic 
flexibility for the authors. The structure of the LMSs for differing didactic possibilities 
seems to be open too. The impression during the interviews was that the answer to 
the question of openness of the LMSs is a theoretical one - the LMS offers this 
variation, but they weren't used anytime. 

1.3 Teacher userfriendliness. 



Three interviewed persons declared the userfriendliness for teachers and course 
developers as "has to be improved". The other stated short training periods between 
four hours and two days. All in all one can say that there should be no difficulties in 
using the LMS. Some statements apply this to word-processing software, internet 
basics or an interface for the LMS. Two institutions needs additional qualification for 
the teacher: competence for studio teaching and certification as telecoach. 

1.4 Support for graphics, audio and video, moving image 

The support for graphics, audio and video, moving image seems to be problem-free. 
The majority declared "all media are supported". But that doesn't mean that all media 
are used but marked the potential of the various LMSs. Two interviewed people said 
that all can be integrated that is internet-capable or java-capable. 

1.5 Questioning, assessment, assignments 

In seven cases there are performance control systems for the students themselves in 
differing variations for example pre-correction, automatic correction or self-testing 
procedures. All other statement can't be summarized. There are differing types of 
questioning (email-based too), established homework-areas, case-studies and 
multiple-choice questions. Also entrance-test, newsgroups and chat are mentioned in 
this area. This mix seems to be e result of the differing "objects" the LMSs is used in - 
to establish a virtual university or a supplement to face-to-face lecturers, to offer 
further education online or specialised courses. There is not one LMS using an 
integrated examination procedure. 
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2 Student support tools 



Institution 


LMS 


studs 


cou 

rses 


tutor 

s 


2.1 

interactivity 


2.2 student to 
student 


2.3 student 
to tutor 


2.4 resources 


2.5 feedback 


Virtus, 

Cologne 


SD 


12 000 


50 


30 


all provided 


news server, no 
chat 


async.24/7 


linked 


self-control 


Teleakad 


BlackB. 


500 


10 


50 


all provided 


all, incl. chat 


textbased 


none extra 


conunents 


Netzentwurf 


SD 


100 


4 


5 


all, databased 


all is too much 


e-mail 
/video conf. 


links 


peer control 


VirtFHS 


Black-B. 


1000 


130 


40 


all via portal 


all possible 


sync= 

scheduled 


possible 


tutor 


LVU Hagen 


(SD) 


27 000 


270 


100 


fixed structure 


newsgroups 


asynchronous 


database 


on/ofOine 


media- 

design 


Leamlink 


700 


5 


130 


all provided 


live 


telephone 

/e-mail 


online,pdf 


on/ofifiine 


Osnabr.VC 


SD 


80 


3 


8 


workgroups 

etc 


asynchronous 


face to face 
-t-async 


WWW 


automated 


VU 

Regensb. 


SD 


280 


57 


35 


newsgroups 


open,incl.chat 


all integrated 


WWW 


not special 


SGD 


SD 


800 


2 


10 


all possible 


all possible 


most possible 


anything 


on/offline 


TUC.VDE 


SD 


1000 


5 


20 


lab. scheduled 


asynchronous 


e-mail 


WWW 


offiine 


asp 


SD 


1700 


60 




m 


asynchronous 


asynchronous 


links 


on/ofiOine 


IPTS 


ecp 


1000 


350 


130 


all possible 


scenarios 


live 


MS,intemal 


good feedback 


SAP AG 


SAP-LS 


20000 


500 


10 


all, intranet 


e-mail 


live, e-mail 


inclusive 


live feedback 


Akad.U. 


SD 


112 


24 


5 


provided 


sync + async 


e-mail,chat 


none extra 


tutor 


VHOberrh. 


clix campus 


200 


6 


n.a 


all (textbased) 


all 


textbased 


In/extemal 


customized 


ZWH 


DLS 


n.a. 


39 


>110 


all possible 


all 


scheduled 

+async 


internal 


within 24hrs 


BildungPlus 


CorpLeam 


n.a. 


25 


40 


all 


sync /async 


24/7,+sync 


all 


not automat. 



2.1 Interactivity possibilities 

The statements regarding interactivity can be divided into three groups: main focus of 
interactivity is the communication with others, the working with the content and a 
combination of both - communication and content. 

Interaction as communication was mentioned in nine cases. That include all activities 
regarding learning groups, message systems, internal and external email-functions, 
newsproups, chat and newsforum. 

Rather content-based interactivities were stated by four interviewed persons - e.g. diaries, 
self-testing, remote-controllable laiboratory, 'Try ist yourself function. 

In another four cases communiction as well as content-based activities are stressed 
as interactivity. 

2.2 Online student to student communication 

For the student to student online communication the following distribution is found: in 
seven cases only asynchronous communication, in five cases only synchronous 
communication and in six cases synchronous as well as asynchronous 
communication is used. 

Asynchronous communication can be done by email, messaging systems, message 
boards, blackboards, mailing lists and so on. The main type of synchronous 
communication is done by chat on different levels - and sometimes on extra channels 
-, net-meetings and group-learning. One institution had offered synchronous 
possibilities but has given it up ..because of the lack of users - the target group 
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memebers study parallel to their respective jobs and don't want to be hooked into 
fixed dates. 

These results describe mainly the used tools, not all provided by the respective LMS. 

2.3 Online student to tutor/institution communication 

Communication between student and tutor/institution is mainly settled in analogy to 
the “student to student communication”. The asynchronous form is available 24 hours 
a day. 

Some special conditions have; to be mentioned. One system contains a feedback 
button for sending messages/nriails to the system administration (as a global adress). 
Another LMS has a virtual-classroom-chat with integrated whiteboard functions and 
synchronous use depends on scheduling. In two cases communication outside the 
system is used - via mail or telephone. 

2.4 Resources, library, references 

The student's acquisition of resources is provided in all surveyed LMSs. The 
resources may be subdivided into access to library, links to additional documents 
and instructional materials, links within the courses and other external links. The 
subgroup "links within the courses" may contain external links too. The table presents 
a summary of the access to the various resources. 



Table of resources, libraty and references 



Institution 


links to library 


links to documents 


links within 


external links 


Virtus 


X 




X 


X 


Teieakad. 


X 


X 






Netzentwurf 


X 


X 




X 


VirtVHS 


X 






X 


LVU 


X 




X 




media design 




X 






UOsnabr.Virt Campus 


X 


X 






Virt.UniRegensbg. 


X 




X 


X 


SGD 


X 


X 


X 


X 


TUC.VDE 






X 




akademie.de.asp 






X 




IPTS 




X 






SAP 


X 


X 






AKAD Oberl 


. 


- 


• 




VirtHS Oberrh 






X 


X 


ZWH 




X 




X 


Biidunq Plus 


P 


E 


p 


p 



x= used; p= possible 



One LMS has a special literature-button which connects with external references e.g. 
digital libraries. In another LMS a commercial information service (EBSCO) is used. 



2.5 Feedback on work and assignments 

In the majority of cases feedback is given by the statements of the tutors. Automatical tests 
are possible in four cases. Three institutions declare that feedback belongs to the content 
and because of that into the responsibility of the authors. Two of them offered automatical 
test - the authors can decide whether they use a more elaborated form of feedback. Another 
mentioned possibility is described generally as "using communication tools". 

The LMS "VIRTUS" doesn't provide any feedback - "VIRTUS" is a supplement to the 
exercises area of face-to-face study. In the beginning feedback on students 
behaviour was evaluated, user-tracking too, but it was stopped. 
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In the LMS ''Netzentwurf' feedback is organized by publishing the steps of work and 
the final product - the architectural design. That means critique from other people is 
the regular feedback - "the main didactic element built in the system". 



Table : feedback on work and assignment 



Institution 


Using 

Communication 

tools 


tutor 

statements 


automatical 

tests 


depends on 
teacher 


Virtus 




- 


- 


- 


Teieakad. 




X 






Netzentwurf 


1 








VirtVHS 






X 


X 


LVU 






X 


X 


media design 




X 


X 




UOsnabr.Virt Campus 


X 








Virt.UniRegensbg. 


X 








SGD 




X 


X 




TUC.VDE 




X 






akademie.de, asp 




X 






IPTS 


X 








SAP 




X 






AKAD Uberl 










VirtHS Oberrh 




X 






ZWH 




X 






Bildung Plus 








X 



ERJC 
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3. Tutor support tools 



Institution 


LMS 


studs 


cou 

rses 


tutor 

s 


3.1 

tracking 


3.2 groups 


3.3 

questions 


3.4 monitoring 


3.5 admin 


Viitus, 

Cologne 


SD 


12 000 


50 


30 


separated 


rights 

assigned 


open 


anonymized 


separated 


Teleakad. 


BlackB. 


500 


10 


50 


by tutor 


tutor instals 


e-mail 


by tutor 


same tools 


Netzentwuif 


SD 


100 


4 


5 


by address 


shared 

workspace 


normal 

drop-out 


weekly report 


open platform 


\TutFHS 


Bladc-B. 


1000 


130 


40 : 


no database 


groupware 


question 

pools 


credit accoimt 


tutor"course" 


LVU Hagen 


m 


27 000 


270 


100 


directory 


by enrolment 


by scheme 


per course only 


Wd) Assign 


media-design 


Leamlink 


700 


5 


130 


complicated 


build 

subgroups 


not optimal 


any questions? 


telephone 


Osnabr.VC 


SD 


80 


3 


8 


mtor=sysad 


groupware 


assignments 


by 

comments 


separated 


VU Regensb. 


SD 


280 


57 


35 


enrolment 


newsgroup 


n.a. 


n.a. 


telephone 


SGD 


SD 


800 


2 


10 


databased 


newsgroups 


not user 
friendly 


tutor 


trackable 


rUC,VDE 


SD 


1000 


5 


20 


elaborated 


groupware 


tutor version 


monthly 


tutor=sysad 


asp 


SD 


1700 


60 


- 


data protect 


groupware 


Mutiple 

choice 


one-to-one 


much possible 


EFTS 


eq> 


1000 


350 


130 


logs 


role assigning 


only 

synchron. 


live 


separated 


SAP AG 


SAP-LS 


20000 


500 


10 


databased 


studs register 


mock exams 


logs 


integrated 


Akad.U. 


SD 


112 


24 


5 ; 


log time 


extra groups 


to be 

developed 


time oriented 


sysad control 


VHOberrh. 


clix 

campus 


200 


6 


n.a. 


n.a. 


groupware 


uploaded 


logins 


not included 


ZWH 


DLS 


n.a. 


39 


>110 


log times 


groupware 


forms offered 


lop 


tutor=responsi 

le 


BildungPlus 


CorpLear 


n.a. 


25 


40 


logs 


groupware 


not much 


logs 


not much 




3.1 Tracking students - database questions 



The user-friendliness of tracking students is described as very easy as well as very 
complicated. The range of information, which tutors can notice differs enormously: 
names and logins; names, credits and assignments; enrolled courses, overview on 
each student; done work and results; number of registered students; logins and 
accessed learning units; taken down learning process and login-times and number of 
actually enrolled students. Another difference is the right to access the database - if 
available. In six cases tutors explicitly have no right to access, only the 
systemadministrators can do so. 

There cannot be found a general setting: it seems to depend on the function of the 
offers - e.g. supplement to face-to-face study or further education - and an 
(individual) interpretation of data protection law. 

3.2 Group management tools 

The most simple way is to manage a group of students who are formally enrolled in a 
course. The allocation of these students is not the task of the tutors. Within the 
groups there are various rights to administrate the courses, including the right to 
build subgroups. This happens in many cases by groupware tools with a lot of 
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cooperation possibilities. In some LMSs group cooperation is organized without an 
assigned tutor. The main element of one LMS is just the possibility of facilitating 
groups. Otherwise there are no special tools but the use of chat, newsgroups or other 
communication tools. One LMS is explicitly described as having "no group product". 

Another LMS differentiates - on the synchronous level - the role of a "learning 
administrator" and a "tutor" (role division). The learning administrator is the person 
who is responsible for setting up a course: students, course leader and course 
material. Tutors are not allowed to do so. 

3.3 Preparation of questions and assignments by tutor 

In this section the answers range from "not user-friendly" to "sophisticated", and not 
applicable. There exist integrated tools which function very well. The range of 
methods are; email, question pools, pre-pared schemes, tools for setting up multiple 
choice questions, uploads and offered forms. In one case the courses contain two 
versions - a tutor's one and a student's one - and they abstain from further support 
because the experts are in the (special) background. 

The main problem in this case seems to be a technical one. The creation of 
questions and assignments without using prepared question pools or schemes or 
other "light" tools depends on a good technical qualification of the course creators. In 
the words of an interviewed system administrator: "It is extremely user-friendly for our 
staff but extremely difficult for external people". 

3.4 Course planning for students (monitoring pace) 

Monitoring students is realized in various forms and intensity. One has to 
differentiate between monitoring a single course or monitoring students over a longer 
period - for instance more than one course. In the case of one course monitoring this 
is realized in forms as 

• weekly reports, 

• general questioning "Any question?", 

• dispatching the teaching material monthly - like a timetable , 

• logs and logins. 

Also comments on exercises and monitoring one-to-one are mentioned. 

For the other form - monitoring over a longer period, for instance university studies - 
the practise is extremely different too. The most elaborated form is the credit-account 
as a good overview (exams are done face-to-face). Another LMS only uses 
anonymized monitoring out of data protection reasons. For another LMS it is stressed 
that monitoring over a longer period is not possible, because the tutor is responsible 
only for his course and can only see his students. This depends on the 
organizational structures of universities - there is an examination board collecting the 
results of student's work. And this examination board is not part of the LMS. 

3.5 User-friendly administrative systems between tutor and institution 

In addition to the basic tools for communication the possibility of installing extra 
courses for mentors/tutors and the (not used) possibility of setting up shared learning 
spaces are mentioned. 

In some other cases the communication is done by "traditional" media - telephone 
and face-to-face, also if communication tools are available. For other LMSs - mostly 
private organisations - the institutional aspect is stressed: the tutors carry out their 
assigned activities, tutor activity can be traced or the tutor belongs to the institution. 
The direction of communicatiori seems to be very clear. 
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The general impression is that the tutor to institution communication is a less 
important aspect of the LMS. This sub-systems can also be served by traditional 
media, and most of the researched LMSs are easy to understand. 
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4. Administration aspects of LMS 



Institution 

Identification 


LMS 


ft 

stude 

nts 


cour 

ses 


tuto 

rs 


4.1 

enrolment 

procedure 


fees 


4.2 

pw/ssh 


4.3 

records 


4.4 exam 
database 


4.5 tutor 
db 


Virtus, Cologne 


ILIAS 

(SD) 


12 

000 


50 


30 


Uni* * 

enrolment 


nU 


pw 


manual 


manual 


admin= 

open 


Teleakademie 

Furtwangen 


Black* 

board 

BSCW 


500 


10 


50 


manual 


nU 


pw 


access 


n.a. 


access 


Netzentwurf, 

Karlsruhe 


SD 


100 


4 


5 


Uni- 

eiu'olmentl 


nil 


pw 


documents 

kept 


manual 


aa 


VirtueUe FHS 
Luebeck 


Black- 

board 


1000 


130 


40 


Uni- 

enrolmentl 


nil 


pw 


manual 


grade- book 
not 

integrated 


UNIX links 


LVU Hagen 


VU(SD) 


27 

000 


270 


100 


Uni- 

eiurolmentl 


nil 


pw 


manual 


plaimed 


integration 

planned 


media-design, 

Munchen 


Leamlink 

evoeye 


700 


5 


130 


eiurolment 
for 1 year 


EURO 

15.000 


pw, ssh 


integrated 


separated 


EXCEL 


Uni Osnabrueck 
Virteller Campus 


VC 

Prolog 

Tutor 

(SD) 


80 


3 


8 


Uni- 

eiurolment 


nU 


pw,ssh 


n.a. 


n.a. 


personalize 

d 


VirtueUe 

Universitat 

Regensburg 


planetux 

(SD) 


280 


57 


35 


Uni- 

eiuolment 


nil 


pw 128 
bit 


manual 


n.a. 


form based 


Studien- 
gemeinschafl 
Darmstadt (SGD) 


wave 

learn 

(SD) 


800 


2 


10 


inter&ce 


ca2000 
Euro/ year 


pw 


no 

problem 


possible 


included 


TU Chemnitz, 
VDE 


(SD) 


1000 


5 


20 


external 


nil 


pw, ssl 


xml 

databases 


possible 


included 


akademie.de asp 
GmbH 


(SD) 


1700 


60 


non 

e 


online 


Euro 150 


pw, ssl 


database 


possible: 
partners job 


possible 


EPTS Kronshagen 


interwise- 

eq) 


1000 


350 


130 


aU online 


nil 


pw 


logs 

possible; 
not used 


not used 


possible 


SAP AG 


SAP 

Learning 

solution 


2000 

0 


500 


10 


all online 


internal 


pw, part 
of portal 


integrated 

into 

system 


test engines 
used 


resource 

mangement 

system 


Akademie 

Uberlingen 


lemen - 
im-netz 
(SD) 


112 


24 


5 


e- 

commerce 

online 


online fee 

2.70 

Euro/hr 


pw,ssl 


self eval. 


external 

tests 


flexible 


VirtueUe 

Hochschule 

Oberrhein 


clix 

campus 


200 


6 


n.a. 


online 


nil 


pw 


n.a. 


n.a. 


tutors space 


ZWHDiisseldorf 


DLS from 
ETS 


n.a. 


39 

^ste 

ms 


>11 

0 


online 


by 

institution 


pw 


no 

problems 


presence 

phases 


admin.only 


BildungPlus 
eLearning 
Gesellschafl mbH 
i.G 


corporate 

learning 


n.a. 


25 


40 


online, or 
sent 


by 

institution 


pw 


n.a. 


n.a. 


n.a. 
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4.1. enrolment procedure 

Unlike in other countries we do find many projects, where the enrolment as a 
university student is the only prerequisite for participating in the LMS. This also 
means that students receive this right to participate together with a password into the 
university computing system. But what does it mean for the procedures? Is there a 
link between the LMS and the student database? No, in almost all cases this means 
that the student just has to identify him/ herself and that no additional fees are 
expected. It explicitly does not mean that the LMS is integrated into the normal 
university enrolment procedures; on the contrary, in most cases this will be 
completely separated. The remarks about "double dichotomy" can enlighten the 
situation. This procedure seems to be standard for most public systems in Germany. 
Also this is the reason why additional fees on the side of the student are not 
expected. Then we find one figure 15.000,- Euro in one case where the sum seems 
to be extraordinary; This sum will be paid by the public, the unemployment agency, to 
finance the enrolment. Where we learn about the fees for a single student (to be paid 
per year or semester or even per minute- in one case) we may well esteem the real 
cost of such systems. 

4.2 passwording 

All systems are passworded; half of them even with additional security aspects. So 
administrative bureauaatic arguments would actually not be valid as arguments 
against integrating the LMS ; into the normal administrative procedures of the 
university or agency. 

4.3 students records 



The ability of a high school student to access the schools database of records has 
been ten years ago the triggering scene for the movie "War Games." What seemed 
to be standard procedure in American high schools exactly is the point where 
German systems deviate; the records database is kept - if at all centrally - at a 
different place of the administration than the enrolment database. Because of the 
privacy laws of data protection it is not so easy to change these procedures. So this 
is the reason why we find in our overview table one third of the LMSs stating 
"manual" as the procedure to change such records. 

4.4 exam database 



Actually for this aspect all the remarks concerning records databases are valid, too. 
We can assume, though that all the LMSs off the shelves do have integrated a link to 
an exam database, some of these even with additional aspects of test-generating 
possibilities. But in these cases we learn from our interviews that even wehere there 
are such aspects integrated they are not used to their full extent (e.g. BlackBoards 
grade book "not integrated" or where we find "possible”). 

4.5 tutors database 

A good LMS would have such a databse integrated with cross-refence ability eiher 
for the administrators or even for the tutors themselves. In some cases though, we 
find the answer "possible" ," integration planned" or some other excuse for not having 
it integrated as yet. 
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5. Technology aspects of LMS 



[nstitution 

Identification 


LMS 


Student 

s 


cour 

ses 


tuto 

rsi 


5.1 Server/ 
software 


5.2 client 
software 


5.3 

adaptability 


5.4 Size 


5.5 speed 


Virtus, 

Cologne 


ILIAS 

(SD) 


12 000 


50 


30i 


php / sql 
metatags DC 


browser 


change during 
summer 


no upper 
limit 


0.1 sec 


Teleakademie 

Fuitwangen 


Black* 

board 

BSCW 


500 


10 


50 


back-office MS, 
BSCW 


browser, 28.8 
connection 


open flexible 
system 


reached 
upper limit 


no 

multimedia 


Netzentwurf, 

Karlsruhe 


SD 


100 


4 


5 

i 


MS- ast-server, 

access, 

metatagging 


standard 


Qexible system 


not yet 
upper limit 


normal 


VirtuelleFHS 

Luebeck 


Black- 

board 


1000 


130 


40- 


AMS, SCORM 
metatags 


standard 


flexible system 


not yet 
upper limit 


good 


LVU Hagen 


VU(SD) 


27 000 


270 


100 


two SUN 

Oracle 

PATROL 


standard 


flexible system 


not yet 
upper limit 


good 


media-design, 

Munchen 


Leamlink 

evoeye 


700 


5 


130 


NT server 


provided 

incLsatellite 

dish 


updates 

provided 


staff limits 
only 


excellent 
20xfaster 
than DSL 


Uni 

Osnabrueck 

Virteller 

Campus 


VC 

Prolog 

Tutor 

(SD) 


80 


3 


8 

i 


php, sql server 
, DC / Ariadne 


any browser 


summer 

updates 


none 


0.1 sec, no 
multimedia 


Virtuelle 

Universitat 

Regensburg 


planetux 

(SD) 


280 


57 


35; 


Dell 1GB 
RAM, SUN 4 
GBRAM 


standard 


open for 
changes, 
RXML, 
metatagging 


none 


high, 

adaptable 


Studien- 

gemeinschaft 

Darmstadt 

(SGD) 


wave 

learn 

(SD) 


800 


2 


10 

) 


planned 


browser 


open for 
changes, 
Qexible 


no problem 


soon own 
system 


ru Chemnitz, 
VDE 


(SD) 


1000 


5 


20 


LINUX server 


browser 


open for 
changes 


200 at a 
time 


no problem 


akademie.de 
asp GmbH 


(SD) 


1700 


60 


non 

e 


LINUX 
Apache, SQL 


browser 


Qexible, 

metatags 

possible 


no problem 


no 

complaints 


IPTS 

BCronshagen 


interwise- 

eq> 


1000 


350 


130 


SQL,2GB 

RAM 


browser, 

push 

technology, 

fast! 


flexible 


no problem 


very high 


SAP AG 


SAP 

Learning 

solution 


20000 


500 


10 


SAP standard 


browser 


flexible, 

metatags 

possible 


no problem 


very high. 
Intranet 


Akademie 

Uberlingen 


lemen - 
im-netz 
(SD) 


112 


24 


5 


LINUX, php 


browser,ISD 
N, Headset 


flexible 


tutor 

resources 

only 

problem 


DSL, 2-3 
sec. 


Virtuelle 

Hochschule 

Oberrhein 


clix 

campus 


200 


6 


n.a. 

• 


LINUX, SQL, 
Oracle 


browser, 

plugins 


Qexible 


n.a. 


satisfartory 


ZWH 

Dusseldorf 


DLS &om 
ETS 


aa. 


39 

syste 

ms 


>11 

0 


no problems 


browser, 

plugins,low 

end 


flexible 


no limits 


satisfertory 


BildungPlus 

eLearning 


corporate 

learning 


n.a. 


25 


40 


MS, SQL 


browser. 


Qexible 


no limits 


satisfying 
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5.1 Server/ software 



Microsoft or LINUX is one of the alternatives for server software systems for the LMS; Our 
impression is that all need two central systems, one for the program of the LMS and a 
separate one for an SQL database.- Some of the interviewees did not answer to the point- 
this also might be because of the rather great distances between the interviewees and the 
interviewers who might have misinterpreted the answers because of the reason that they are 
not experts in technology themdselves. Any how, some of the answers also indicate an 
inclination to be aware of the problems of standardization (Dublin Core, Metatagging, 
SCORM). 

5.2 client software 



In this category we can be secure in stating that for most systems normal browsers 
will be sufficient. Some of the projects answered the necessary velocity of data 
connection (ISDN or even better for video-connection e.g.) There is only one 
technically very interesting system which splits the technology aspects; The system 
"media design" offers not only a WWW based portal with all the aspects expected 
from a LMS but also daily satellite broadcastings. This would be much too expensive 
because it is not via public broadcasting systems ( The LVU does have a connection 
to the FernUniversitats broadcasting slot of fortnightly public broadcasts) but in the 
case of mediadesign it is rented from the satellite agency for a couple of hours a day 
(which costs immense sums of money), and received by a satellite dish which is 
professionally installed by the LMS. 

5.3 adaptability 



The unanimous data of LMS appearantly is "adaptability provided". We find only two 
of the reported systems having slightly fixed systems because they state about 
adaptability that they will use summer breaks to adapt the systems. So we can 
assume that this category is felt as a simple necessity LMSs will have to be 
adaptable to changing needs and situations. But this statement again is a rather 
flexible one as are such systems: it does not tell about the time lapse of adaptation 
neither what the number and setup of courses not the number and progress of 
students is concerned. Technically all seems to be possible. LMSs are too young to 
make reports about necessary adaptations to changing structures. 

5.4 size 

LMS unlimited is the result of empirical findings: Only one of the interviewed systems 
reported that an upper limit has been reached and states that this problem will be 
solved soon. So neither the limitation of student numbers nor the limits of number of 
courses provided are real problems for the systems interviewed. 

5.5 speed 

We find very accurate data here; 0.1 sec or even 20 times as fast as DSL or the 
rather slow data 0.1 sec because no multimedia is used as yet. Speed is a category 
of the LMSs interviewed which does not create any problems on the side of the 
systems: We do not know about the speed of the systems viewed from the side of the 
users: As expected the speed provided by the systems is much higher than the 
velocity of learners systems at large. But one expression might be taken as the 
overall evaluation of the speed problem: no complaints! 
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6j Price aspects of LMS 



Institution 

Identification 


LMS 


students 


cours 

es 


tutor 

s 


6.1 cost 
total 


6.2 fees 
annual 


6.3 reduct ? 


6.4 

maintena 

nee 


6.5 

training 


Virtus, Cologne 


ILIAS (SD) 


12 000 . 

1' 


50 


30 


open source 


nil 


nil 


5 hrs/week 


£ 900 for 
externals 


Teleakademie 

Furtwangen 


Black- 

board 

BSCW 


500 ( 

i 


10 


50 


Q.a. 


free 


nil 


1 fulltime 


1 test-week 


Netzentwuif, 

Karlsruhe 


SD 


100 


4 


5 


own 


nU 


nil 


1 fuUtime 


1 day 


VirtueUe FHS 
Luebeck 


Black- 

board 


1000 


130 


40 


80.000 $1 


57.000 $ 


campus 

contract 


3 fuUtime 


two days 


LVU Hagen 


VU(SD) 


27 000 


270 


100 


100.000 € 


ml 


nil 


4 fulltime 


alongside 


media-design, 

Munchen 


Leamlink 

evoeye 


700 


5 


130 


£500 

^ear/studs. 


£800 

/hour(sat) 




2 fulltime 


1 day 


Uni Osnabrueck 
Virteller Campus 


VC Prolog 
Tutor (SD) 


80 


3 


8 


n.a. 


5000 

£(server) 


nil 


46hrs 

/month. 


2 hrs 


VirtueUe Universitat 
Regensburg 


planetux 

(SD) 


280 


57 


35 


Ehiblic 

License 


nil 


n.a. 


1 fuUtime 


4 hrs 


Studien- 
gemeinschaft 
Darmstadt (SGD) 


wave learn 
(SD) 


800 


2 


10 


server,datab 

ase 


nil 


none extra 


2 1/2 
fulltime 


1 day 


TU Chemnitz, VDE 


(SD) 


1000 


5 


20 


self devel. 


nil 


10% of 
fees? 


1 hr /week 


1 hr 


akademie.de asp 
GmbH 


(SD) 


1700 


60 


e 

30.0 

00 


online 


none 


10-40 studs 


40- 

80.0006/ 

month 


2 fuUtime 


IPTS Kronshagen 


intenvise- 

eq> 


1000 


350 


130 


not 

permitted to 
teU 


n.a. 


included 


1 fuUtime 


3 fuUtime 


SAP AG 


SAP 

Learning 

solution 


20000 


500 


10 


self devel. 


n.a. 


per 500 
price- 
grading 


>1 

fulltime 


2 days 


Akademie UberUngen 


lemen -im- 
netz (SD) 


112 


24 


5 


self-devel. 


15.000C 


2,70 /hr 


60% 


40006 

/tutor 


VirtueUe Hochschule 
Oberrhein 


clix campus 


200 


6 


n.a. 


n.a. 


n.a. 


nil 


10.0006 


1 day 


ZWHDusseldorf 


DLS from 
ETS 


n.a. 


39 

syste 

ms 


>110 


n.a. 


n.a. 


none extra 


1 day 


1 day 


BildungPlus 

eLearning 


corporate 

learning 


n.a. 


25 


40 


n.a. 


n.a. 


n.a. 


1 fuUtime 


4 hrs 



6.1 total cost 



We do expect little information in this section out of two reasons: In some cases 
especially in public universities total cost will not be known because it is no category 
to look for. The reason for that is manifold; universities, as stated already in the 
section "double dichotomy" have the task to research and train students in exactly 
such systems so it must be hard to differentiate among cost structures. On the other 
hand with products "off the shelves" we find some cases where the administration 
simply forbade to tell exact data. It might be that they received special prices or have 
the hope to in the future. Only in one case we are given exact data but in most cases 
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we can figure out by calculating the staff cost on the side of the institutions. So when 
we look at the reported figures we will find answers in different categories. Installing 
a complete system mostly meant to buy a new machine and database software, 
which easily sums up to some €100.000 Euro. But the answers of the majority of 
projects will hide these cost behind the statement that it is self-developped, open 
source, or not available ( n.a.) 

6.2 annual cost 



What has been said about installing the system holds true for the second and more 
years to come: these cost are not calculated, because there is no need to calculate 
exactly. And proprietary systems are the only ones who do know exact figures but will 
not tell in all cases: 4 data packages with more or less exact data are provided: We 
now learn that BlackBoard costs 80.000 and 57.000 each year for the campus 
contract in the case of Luebeck. 

6.3 student cost/ reduction 



Almost all respondents state that there is none for the students- the one project which 
is financed by "online cost" from the student is different in not charging anything else 
but online fees: that is where they draw their money from. It seems to be a good way 
to support the systems used, and it does provide a possibility to come to just and 
balanced cost structures for the system: but is is the only project to use this form of 
finacing by micro-payment rather than by enrolment fees or other. 

6.4 maintenance 

If we had no idea what cost structures are involved in running a LMS we would know 
now by the answers in this category: Also here we learn about the range possible for 
different but somehow comparable systems: the range is from 1 hr. a week (which we 
don't believe) over 5 hrs a week up to 4 fulltime with a salary of about €50.000, which 
sums up to more than€ 200.000 each year. It might be a useful task for researchers 
to find out about the average yearly maintenance cost per student but the differences 
among the reported systems are too big to get a sound interpretation. 

6.5 training 



training for teachers and students had not been differentiated in the questionnaire, so 
we find answers for both groups: Amazing the range of investment necessary to train 
people in these systems: between 1 - hour for almost self-explanatory systems and 
three fulltime staff for training (most likely alongside the maintenance tasks) which 
will certainly include a permanent hotline then; the data are so widespread as is the 
scope of differences among the systems. It certainly is hard to tell about the cost for 
training due to the wide range of answers in this question category. 
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7. Further development 

The last question "What features would you like to see included in this LMS in the 
future?" should give a view to future developments of the used IMS's. 

It is nearly impossible to generalize future options for the LMSs. Options mentioned 
several times can be grouped as follows; 

a) The import and export of complete courses, modules or material should 
be extended 

b) The LMS should be “enlarged” 

c) More tools for synchronous communication should be integrated. 

The import/export function seems to be the most important issue. This belongs on the 
one hand to the practical use of import/export of content, but on the other hand the 
lack of available content is deplored. Furthermore it should be easier for authors to 
import their own courses. Connected with a better import/export function is the 
problem of establishing metadata and SCORM/ LOM compatibility. 

The second option -the enlarging of the LMS - means the wish that other faculties of 
the university should take over the LMS. Another position in this direction is the 
improvement of user-friendliness in order to gather more people to use the LMS - be 
it as counsellor or be it as colleague. 

The third option - more synchromous tools/interfaces - aims at better and extended 
possibilities of synchronous communication. 

As further aspects mobile computing and real-time video conferencing are 
mentioned. 

Summarized one can say that questions of standardization and optimized 
possibilities of synchronous communication with different tools will play an important 
role in the future. 



List of conclusions from Germany 

1. There seems to have emerged a new dichotomy that plays a major role in the 
development of web education systems. The dichotomy between academia and 
administration 



2. There is a tradition that a bright computing center does not need to buy programs 
developed by others because the need to buy external programs would question the 
qualification of existing personel in such centers. 



3. The task of such a computing center indeed will necessarily produce a conflict 
between the university's administration which hoped for increase of effciciency and 
decrease of performance cost and the department which always has to look for 
proofs of their respectability tested in - at least- the "own" market. 

4. In the majority of cases there is no course creation with or inside the LMS in fact 
independent of the origin of the LMS - self-developed or purchased. 
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4. In the majority of cases there is no course creation with or inside the LMS in fact 
independent of the origin of the LMS - self-developed or purchased. 

5. There is not one LMS using an integrated examination procedure. 

6. Unlike in other countries we do find many projects, where the enrolment as a 
university student is the only prerequisite for participating in the LMS. It explicitly 
does not mean that the LMS is integrated into the normal university enrolment 
procedures : on the contrary, in most cases this will be completely separated. 

7. A records or test-database Is kept - if at all centrally - at a different place of the 
administration than the enrolment database. Because of the privacy laws of data 
protection it is not so easy to change these procedures 

8. The unanimous data of LMS appearantly is "adaptability provided". 

9. Neither the limitation of student numbers nor the limits of number of courses 
provided are real problems for the systems interviewed. 

10. As expected the speed provided by the systems is much higher than the velocity 
of learners systems at large. 

11. Installing a complete system mostly meant to buy a new machine and database 
software, which easily sums up to some €100.000 Euro. But the answers of the 
majority of projects will hide these cost behind the statement that it is self- 
developped, open source, or npt available (n.a.) 

12. The lack of available content is deplored 

13. Summarized one can say that questions of standardization and optimized 
possibilities of synchronous communication with different tools will play an important 
role in the future 
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Abstract 

This article presents the results from an analysis of online education and Learning 
Management Systems (LMS) in the five Nordic countries. The analysis is based on literature 
review and in-depth interviews with 20 selected Nordic training managers. The analysis 
comprises a broad range of institutions from primary education, secondary education, higher 
education, distance education, and corporate training. 

LMS systems seem to be widely used in Nordic education and there is a clear trend towards 
large-scale online education. The 20 institutions had experiences with 25 different LMS 
systems and 12 of the institutions now have more than 50 online courses. Higher education 
institutions have standardized on a few national student management systems, and they prefer 
LMS-systems developed in the Nordic countries. Among the 25 different LMS systems that 
were identified in the analysis, 16 were of Nordic origin. All other systems were of American, 
Canadian, or Irish origin. The research indicates that ClassFronter, WebCT, FirstClass, and 
BlackBoard seem to be the most used LMS systems. E-leaming standards do not seem to have 
had much impact on online education in the Nordic countries. 

LMS systems could have reached a point were user-fiiendliness, cost effectiveness, and 
integration with other systems is more important than new features. Some interviewees want to 
integrate the LMS with existing systems and other services such as student management 
systems, marketing catalogues, online payment, tracking of textbook shipments, registration of 
examinations, and multimedia tools. 

The institutions do not seem to be especially loyal to, or dependent on, one provider of LMS 
system. Several institutions prefer self-developed systems. They perceive the commercial 
systems as expensive and complex and want to develop the systems to support their special 
needs. They wanted cost effective systems with the ability to handle continuous enrollment and 
integration with student administrative systems and economy systems. In the future, the open 
source strategy may have an impact on the LMS market. 

introduction 

This article presents the results from an analysis of online education and Learning Management 
Systems (LMS) in the Nordic countries. A more comprehensive and detailed analysis (Paulsen 
2002) is available in the complete, one hundred page report. The analysis is an integral part of 
the European Web-edu project (www.nettskolen.com/in_english/webedusiteAmdex.html) that 
provides similar analyses from other regions in Europe. The Nordic analysis is based on 
literature review and in-depth, qualitative telephone and e-mail interviews with 20 selected 
Nordic training managers who have comprehensive experience using LMS systems. The 
interviews were conducted from October 2001 to May 2002 according to an interview guide 
developed by the Web-edu project team. The interview guide identified the following focal 
points, which also are discussed in this article: 
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• The institutions and their LMS systems 

• Course development tools 

• Student support tools 

• Tutor support tools 

• Administrative systems 

• Technology 

• Economic issues 

• Overall evaluation 

• Features in future LMS : Systems 



The Nordic Scene 

The 5 Nordic countries are Denmark, Finland, Iceland, Norway, and Sweden. Sweden has a 
little more than 8 million inhabitants, Denmark and Finland a little more than 5 each, Norway 
4.5 millions and Iceland about 300.000 inhabitants. Together they have a total population of 
about 23 million. Each country is among the wealthiest nations in the world. The countries are 
also recognized as advanced users of computers and telecommunication technology. According 
to the key indicators presented in eEurope’s Benchmarking program 
(http://europa.eu.int/information society/eeurope^enchmarking/index en.htm\ all five 
Nordic countries are among the six EU countries that have the highest Internet access in 
households. 



The following paragraphs present an overview of the situation regarding national initiatives on 
online universities, student management systems, and LMS systems in each of the five 
countries. The issues are further discussed in the articles Online Education Systems in 
Scandinavian and Australian Universities: A Comparative Study (Paulsen 2002) and Online 
Education Systems: Discussion and Definition of Terms (Paulsen 2002). 



Denmark 

According to Ranebo (2001) the Danish Ministries of Education and Research initiated a 
Danish Virtual University in a mission statement on March 27, 2000. The DKr 40 million 
budget for the period 2000-2003 should be used to support the development of high quality, 
higher education, web-based courses and provide information about the courses. As one of the 
interviewees pointed out, this was obviously not a success: 



The universities have autonomous responsibilities for their e-leaming strategies. A national 
initiative to establish a Danish Virtual University broke down as a result of disagreement 
between the involved partners. The only result seems to be a planned portal providing 
information about e-leaming initiatives. 



STADS is the dominant student management system in Danish Universities and colleges. All 
Danish universities except firom the universities of Copenhagen and Aarhus use it. The system 
is developed by WM-Data in collaboration with the universities. 



Scandinavia is an important market for FirstClass and some of its largest customers are in 
Denmark. Skol-kom has more than 200.000 FirstClass users in Denmark. COM-C and 
BlackBoard also seem to be important players in the Danish market according to two 
interviewees: 
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Blackboard makes advances in Denmark. 



BlackBoard seems to grow in the Danish market. COM-C and FirstClass are two 
alternative, major players. 

Finland 

According to Ranebo (2001), the 20 Finnish universities have formed the Finish Virtual University 
(www.virtuaalivliopisto.fi) as a consortium: 

The Virtual University of Finland is a development project that was initiated by the Finnish 
Ministry of Education during the year 2000. The project is a step in the realization of the 
overall strategies that the Ministry of Education presented in 1999 in its Knowledge 
Strategy for Education and Research 2000-2004. 

The aim is for a virtual university, which offers Web-based high standard courses also in the 
international level, to be set up by the year 2004. Through co-operation involving 
universities, colleges of higher education, research centers and private alternatives the 
quality and flexibility of training will be reinforced, and research networks will also be 
reinforced. 

The 20 universities in Finland will form the foundation of the activities and so, in January 
2001, they established a consortium. Students who study on a full or part-time basis at the 
virtual university must be officially accepted at a physical university if they want to get a 
degree. University students do not have to pay for their studies in accordance with Finnish 
legislation. However, the universities can also sell courses and commission courses on the 
open market and thus provide courses with tuition fees for the international market, for 
example. (Ranebo 2001) 

Oodi seems to be the dominant student management system in Finish universities. But it does 
not seem to have such a dominant position as the SMS systems in the other Nordic countries. 

One interviewee states: 

There are 21 universities in Finland and there are 11 different LMS systems. 8 of the 21 
universities use the so-called "Oodi" SMS system, but only 5 of them have Oodi in real use. 

Oodi is based on the Windows and Oracle (+Uniface + WebLogic). You can find more 
about Oodi at the following address, but unfortunately it's available only in Finnish! 
www.oodi.fi . 

The researcher found an Oodi manual in English via a Google search at 
httD://atk.hkkk.fi/english/manuals/oodiohie eng.htm 

Iceland 

According to Ranebo (2001), Iceland has not made any decision to establish a national virtual 
university: 

In Iceland, no overall decision has been taken by the state to assemble all the higher 
education efforts in the area of distance learning under a national development programme 
for Virtual University or Web University. This particular area in higher education is 
relatively broad in Iceland, and distance learning, as a development area is a matter of top 
priority politically. Due to the size of Iceland, it is fairly easy to gain a fairly good insight 
into what is on offer nationally when it comes to distance courses. This might be one reason 
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why there has not been a need to create a national comprehensive organization. (Ranebo 

2001) 



The Ministry of Education requires the schools to apply a student management system named 
INNA. One interviewee states: : 

Up till now, we have used a student management named AXEL, which was developed in 
Iceland. Every school had an individual installation of the same system. Now, the ministry 
of education requires that all schools use a central web-based system called INNA 
(www.mna.is). A company called SKYRR (www.skyrr.is) has developed INNA for the 
ministry of education. Neither AXEL, nor INNA has any integration to WebCT and I 
cannot see that WebCT will be integrated with INNA in the future either. 

Norway 

There is no national online university in Norway, but a number of public initiatives that support 
and fund online education in existing institutions. 

In Norwegian higher education, the dominant LMS-system is the Norwegian-developed 
ClassFronter (www. fronter . coml . Some colleges use standard commercial systems, and some 
institutions have developed their systems in-house. Runnesto and Ristesund (2002, 36) 
surveyed 54 of Norway’s universities and colleges. Out of these, 32 institutions claimed that 
they offered online education. ;Several of them had used more than one system as shown in 
Table . 



Table 1. Instances of LMS systems in 54 Norwegian institutions of higher education 
(Source: Runnesto and Ristesund, 2002) 



LMS 


Number of instances 


ClassFronter 


21 


In-house developed 


9 


LUVIT 


6 


IT’s Learning 


4 


First Class 


3 


Kark 


3 


WebCT 


3 


BlackBoard 


2 


TopClass 


1 


Lotus Learning Space 


1 


TeamWave 


1 


Response 


1 




In Norway, two student management systems totally dominate in higher education. The 
Norwegian universities and some colleges use FS (Felles System) ( http://www.fs.usit.uio.noA 
and most of the Norwegian colleges use MSTAS (www.enet.no/) . The two largest private 
colleges have chosen alternative solutions. The Norwegian School of Management BI has 
experiences with Banner and NKI has developed an in-house system called STAS. 

Sweden 

In Sweden, no LMS-systems seem to be dominant, but a number of standard commercial 
systems are used. Two of the comments made by the interviewees support this: 
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The Swedish developed LMS systems are not dominant in Swedish education. There is little 
national coordination in this field, the universities are very autonomous and the system 
choices are made locally. 

Most of the distance education courses we provide via LUVIT are included in the national 
network university program, Ndtuniversitet (www.netuniversity.se). It allows the 
universities to apply their own online systems and models. In other words, the universities 
don’t need to coordinate their systems or pedagogical models. This is probably good for the 
universities, but it could be confusing for students who want to follow courses from several 
universities. 

According to personal e-mail communication with Fredrik Rexhammar (March 18, 2002), one 
of Sweden’s leading experts on LMS-systems, Luvit (www.luvit.com') . Lecando, Infinity, 
Grade, Platon, and Web Academy are Swedish providers of LMS systems. In the same e-mail 
he states that WebCT and Blackboard are the most used foreign LMS systems used at Swedish 
universities and colleges. In addition, the author has found that the Swedish-developed system 
PingPong (www.partitur. se~) is used by some institutions. An overview of the market for LMS 
systems conducted in Sweden is available at www.ssv.gov.se/avit/pfomi2.htm . It lists the 
following systems: Luvit, Mentor, Telia Instant Education, Maestro, FirstClass, Comenius 
online, Leicando, Librix, Marratech, PingPong, Surfa och lar, and WebCat. 

The three Swedish distance education consortia have received considerable governmental 
funding since 1993-94. (Hillefors et al, 22 and Ranebo 2001). At most, the three consortia 
offered 40-50 courses to 5000-6000 students (Hillefors et al, 26). After nearly ten years of 
operation, their results are not impressive and their funding will be discontinued. Instead, the 
government has recently established Ndtuniversitetet fhttp://www.netuniversitv. se") as a new 
national body to fund and coordinate the activities. In 2002, Ndtuniversitetet will provide 
financial funding for the equivalent of 2 350 full-time students at 30 Swedish higher 
institutions. 

All Swedish universities are using the student management system LADOK or LADOK 
NOVAU that are owned by a consortium of 37 institutions in higher education in Sweden. The 
LADOK consortium (http://www.ladok.umu.Se/l provides the following information at its web 
site: 

LADOK is a computer based student admission and documentation system for a university 
or university college. It focuses on administration of undergraduate and graduate students. 
The system is locally deployed and managed by the institutions.... The LADOK-system 
consists of two major parts, the admission system and the documentation system. They are 
integrated and share data, e.g. name, address and other facts about applicants and 
students.... The system files contain information for student identification, general eligibility 
for university studies, admission to courses and study programs, registration on courses per 
semester, course data, credit points from courses, awarded degrees and international 
studies. . . . Data from LADOK are exported to the ministry of education and other agencies 
for follow-up purposes. An important objective of LADOK is to prepare the annual invoice 
to the government for studies on the undergraduate level at an institution. . . 
(www.ladok.umu.se/opendok/LADOK_short.html') 



Overview of the Institutions and their LMS Systems 

The analysis comprises a broad range of institutions from primary education, secondary 
education, higher education, distance education, and corporate training. But about half of the 
institutions are characterized as public universities. It includes public and private institutions as 
well as both providers of LMS services and costumers of LMS services. 



Table 2 shows that the analysis includes 20 institutions from all Nordic countries. Five of them 
are Danish, four Finish, one Icelandic, five Norwegian, and five Swedish. 

Table 2. Institutions Sorted by Country 



Damnarks Netskole 


www.netskole.dk 


Denmark 


Type of Instittitioa 

Consortium of public technical colleges 


Center for 

Fjemimdervisning 


www.cfu.dk 


Denmark 


Distance education center at business 
college 


The Centre for the 
Interdisciplinary 
Study of 

LeamingAalborg 
University 


www.vcl.auc.dk/default- 

eneelsk.htm 


Denmark 


University centre 


Ventures 


www.ventures.dk 


Denmark 


Consortium 


University of 

Southern Denmark 


WWW. sdu. dk/indexE . html 


Deiunark 


Public university 


University of Oulu 


www.oulu.fi/en2lish/ 


Finland 


Public university 


University of Art and 
Design Helsinki 


www.uiah.fi/enelish.shtml 


Finland 


Public university 


University of Kuopio 


www.uku.fi/enelish 


Finland 


Public university 


University of 

Tampere 


www.uta.fi/enHlish/index. 
html ^ 


Finland 


Public university 


Comprehensive 
College in Akureyri 


www.vma.is 


Iceland 


Vocational college with a distance 
education department 


The Competence 

Network 


www.nkn.no 


Norway 


Commercial provider of LMS-related 
services 


Nettskolen 


www.nettskolen.no 


Norway 


Commercial provider of courses 


Nettgymnas 


www.nettevmnas.no 


Norway 


Private secondary school 


Globalskolen 


WWW. elobalskolen.no 


Norway 


Semiprivate provider of primary 
education for children abroad 


NKI 

Fjemimdervisningen 


WWW. nettskolen. com 


Norway 


Private distance education institution 


Midthdgskolan 


www.mh.se 


Sweden 


Public university 


Statens skolor fbr 
vuxna 


www.norrk. ssv. se 


Sweden 


Public distance education institution 


Skandia 


www.skandia.com 


Sweden 


Global savings company 


University of 

Uppsala 


www.uu.se 


Sweden 


Public university 


University of Lund 


www.lu.se 


Sweden 


Public university 



Among the 25 identified LMS systems, 1 1 are listed as “other LMS”. This means that it is not 
the primary LMS system used at the institution at the moment. It could be a system that has 
been used in the past, a system that is tested for future use, or just a secondary system used for 
special purposes. It is interesting to observe that Table 3 shows that 1 1 of the 20 institutions 
have experiences with one or more “other systems”. This indicate that the institutions are not 
especially dependant on, or loyal to, their providers. This situation is both a result of local 
autonomy and historic development. 

Table 3 shows that there are large variations among the institutions with regard to 
the number of online courses, tutors, and students. 
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The numbers of online courses range from 850 [NKN] to 4 [Skandia]. The high number of 
courses listed by NKN is explained by the fact that it includes courses from 50-60 course 
providers. There are however eight institutions that claim to ofier more than 100 online 
courses. 

The numbers of online tutors range from 10 or less [Nettgymnas and Globalskolen] to 
hundreds. Four institutions claim to have more than 100 online tutors. [NKN, NKI 
Fjemundervisningen, Midthogskolan, and Aalborg] 

The numbers of online students range from a few hundreds to several thousands. Even though 
the numbers regarding online courses, tutors, and students not necessarily are comparable and 
reliable, one may conclude that most of the institutions offer online education in a large scale. 
If one characterizes institutions that offer at least 50 online courses as large-scale providers of 
online educators, 12 of the 20 institutions are characterized as large-scale providers of online 
education. In a previous international analysis of web-based education (Paulsen 2000) only 3 of 
the 22 Nordic institutions surveyed in 1998-99 offered more than 50 online courses. The 
analysis indicates that there is a clear trend towards largescale online education in the Nordic 
countries. 

Table 3 also shows that there are substantial variations in the number of years in use and 
course duration. The number of years in use range from less than one year to 15 years [NKI 
Fjemundervisningen]. Eight institutions claim to have up to 2 years experience. Only six 
institutions have five or more years of experience. 

The typical course duration seems to be several months. This is not surprising since most of the 
institutions in the analysis are educational institutions that traditionally offer longer educational 
programs to students, not shorter courses to companies. 



Table 3. Information about Surveyed Institutions Sorted by LMS 



i'Naineof' 


LMS . ; ' 


:ChHeri'-''Lr 








M ■ 


Typical 


Institution. 






•Oidiiiie:'. 


iOniind i; 


iStudimtsiiMh 


::.:ytorS 


Course 








iCbtirseS: 


Tutors ^1;- 




Rri'iUse 




University of 
Southern Denmark 


BlackBoard 


BettyCom, 

EDWIN, 

FirstClass, 

COM-C 


110 


50-60 


240 in the 
department 
and 15000 at 
the university 


! 7 




Nettskolen 


ClassFronter 

CourseKeeper 




10 


15 


150+150 


1 2 


2-3 months 


Nettgymnas 


CourseKeeper 




6 


6 




<1 


1 school 
year 


Globalskolen 


FirstClass 


PedIT 

Imaker 


2S 


10 


180+150 


1 1 


1 school 
year 


The Centre for the 
Interdisciplinary 
Study of Learning, 
Aalborg University 


FirstClass 


Virtual-U 


11 

program 

s 


120 


SSO 


I s 


1 semester 


Ventures 


FirstClass 


None 


A 

number 

of 

program 

s 


50 


3000 


1 5 


Typically 

on 

semester, 
but also 
offer 

individual 

enrollment 


University of Art 
and Design 
Helsinki 


Fle3 - Learning 
Environment 


WebCT 


20 






4 


4-16 weeks 



liz 



mmm 
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Name of 


LMS 


LMSr 


# ■ ••• 

Otdme . 
iCoarses 


#■ ; 

OiiSne: 
Tutors ' 


# Online \ 
Stucients 


:# 

Years 
'in Use 


Typical : 

Cmirse' 

Duration 


University of Oulu 


LC Profiler 




41 


??? 


2500 users 




Several 
weeks or 
months 


Skandia 


Luvrr 


None 


4 


??? 


650 


2 


6 months 


University of Lund 


Luvrr 

i 

1 


Lotus 

Learning 

Space 


250 




6000 of the 
23000 
students are 
LUVn users 


5 




University of 
Uppsala 


Ping Pong 


FirstClass 


50 


50 


20000 
personal 
accounts, 
2000 real 
users 


1.5 


0.25-0.5 
school year 


NKN 


Saba 


Wd)Leam 

Plus 


850 


hundreds 




1 


3 hoiu:s - 6 
months 


Danmarks 

Netskole 


Self-developed 




40 


30 


7000 coiuse 
enrollments 


<1 


16 weeks 


NKI 

Fjemundervisning 

en 


SESAM(Self- . 
developed) 




250 


125 


3300 


15 


6 months 


SSVN 


SSVN2000 

(Self- 

developed) 


LEKTOR 


50 


20-25 


500-1000 


1 


Few weeks 

-several 

years 


CFU 


TopClass 


BlackBoar 

d 


20 


Equivale 

ntof4 

full-time 

positions 


500 


4 


1-4 months 


Midthdgskolan 


WebCT 


FirstClass 

West 


200 


Between 
100 and 
200 


2-3000. 40% 
at a distance, 
60% on 
campus 


5 


0.5-4 

semesters 


University of 
Kuopio 


WebCT 


Verkkosalk 

ku, 

Verkko- 

opisto 


200 




4500 


3 


40-160 

hours 


University of 
Tampere 


WebCT 




140 




3700 


3 




Comprehensive 
College in 
Aukureyri 


WebCT 


None 


180 

online 

courses, 

10-20 

apply 

WebCT 


95 


750 


4 


1 semester 
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Table 4. LMS Systems Sorted by Original Nationality 



IMSsystienis ; 


Onlgiiul ' 

NatfonaB 

ty 


Lai^iiageof LMS 


imLofLMS 


lAstitiitioos Bsii^ I!M5 ! 


Intftadons ssing 
C^erLMS 


Webleam Plus 




English 






NKN 


West 










Midth&gskolan 


BlackBoard 


American 


English, Danish 


www.blackboard.com 


Southern Denmark 


CFU 


Lotus Learning 
Space 


American 




www.lotus.com 




Lund 


Saba 


American 


English, Norwegian 


www.saba.com 


NKN 




FirstClass 


Canadian 


Norwegian, Danish, 
Others 


www.firstclass. com 


Globalskolen, Ventures, 
Aalborg 


Midthdgskolan, 
Southern Denmark, 
Uppsala 


Virtual-U 


Canadian 




www.vlei.com 




Aalborg 


WebCT 


Canadian 


English, Swedish, 
Not available in 
Icelandic 


www.webct.com 


Midthdgskolar4 Kuopio, 
Tampere, Akureyii 


UIAH 


BettyCOM 


Danish 








Southern Denmark 


COM-C 


Danish 








Southern Denmark 


EDWIN 


Danish 








Southern Denmark 


Self-developed 


Danish 


Danish 




Danmarks Netskole 




Fle3 


Finish 


Finnish, Swedish, 
English, Spanish, 
French 


httD://fle3.uiah.fi/ 


UIAH 




LC Profiler 


Finish 


Finish, English 


www.lcprofcom 


University of Oulu 




Verkkosalkku, 

Verkko-opisto 


Finish 








Kuopio 


TopClass 


Irish 


English, Danish 


WWW. wbtsvstems. com 


CFU 




ClassFronter 


Norwegia 

n 


Norwegian, English 


www.fronter.com 


Nettskolen 




CourseKeeper 


Norwegia 

n 


Norwegian, English 


www.coursekeeper.co 


Nettskolen, Nettgymnas 




Imaker 


Norwegia 

n 




www.imaker.no 




Globalskolen 


Self-developed: 

PedIT 


Norwegia 

n 


Norwegian 






Globalskolen 


Self-developed: 

SESAM 


Norwegia 

n 


Norwegian 




NKI 

F j emundervisningen 




LEKTOR 


Swedish 


Swedish 






SSVN 


Luvrr 


Swedish 


English, ; German, 
French, Chinese, 
Swedish, Danish, 
Norwegian 


www.luvit.com 


Skandia, Lund 




Ping Pong 


Swedish 


Swedish, English, 
German, French 


www.partitur.se 


Uppsala 




Self-developed: 

SSVN2000 


Swedish 


Swedish 




SSVN 







Table 4 shows that the 20 institutions had experiences with 25 different LMS systems. It is 
interesting to observe that the majority (16 of 25) of the LMS systems are of Nordic origin. All 
other systems were of American, Canadian, or Irish origin. 



The analysis further shows that 3 institutions (Danmarks Nettskole, NKI Fjemundervisningen, 
and SSVN) have chosen to use self-developed LMS systems. One additional institution 
(Globalskolen) states that it will convert to a self-developed system. 



The commercial LMS systems that are most widely used among the institutions in this analysis 
are FirstClass and WebCT. The strong position of WebCT is not surprising, since WebCT and 
BlackBoard presently are the two dominant LMS systems on the international market 



CRJQ 
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(Observatory on Borderless Higher Education, 2002). FirstClass’ European headquarter is 
located in Sweden. Scandinavia is an important market for FirstClass and some of its largest 
customers are in Denmark. Skol-kom has more than 200.000 FirstClass users in Denmark. It is 
also important to know that a comprehensive analysis (Runnesto and Ristesund, 2002) shows 
that the Norwegian system ClassFronter is very dominant in Norwegian higher education. 

The commercial LMS systems are available in several languages, but Nordic institutions prefer 
to use their national languages. This is a special problem for Iceland, since few providers find it 
interesting to develop a version for the Icelandic population of less than 300.000 inhabitants. 
There are also examples of Nordic Institutions that choose to use the English language version, 
since new versions are first released in English. 

Two of the LMS systems in this analysis, Virtual-U and Fle3, are referred to as open source 
systems. The interviewees who point this out are very positive to this, and it will be interesting 
to see if the open source strategy will be able to compete with the other commercial systems. 
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Course Development Tools 

The interviewees have different views on how satisfactory the systems are for course creation. 
But, the LMS systems are usually not used for development of the course content. A broad 
range of external tools is used to develop the content before it is published in the LMS system. 
The interviews show that the LMS systems use text, multimedia, sound, html-pages, graphics, 
and tests that are developed with external software. The software tools for course creation 
referred to in the interviews are listed in Table 5 



Table 5. Software Tools Used for Course Creation 





Typetrfctmtenfil 




URL 


Word 


Text 




PowerPoint 


Text 




Macromedia Authorware and 
Director 


Multimedia 




Flash 


Multimedia 




Windows SoundRecorder 


Sound 




Wimba 


Sound 


www.wimba.com 


Frontpage 


HTML-pages 




DreamWeaver 


HTML-pages 




Netscape Composer 


HTML-pages 




Viewlet 


Graphics 

(Screenshots) 


www.aarbon.com 


Coral 


Graphics 




Photoshop 


Graphics 




ReadyGo 




www.readveo.com 


ToolBook 






Autotest 


Tests 




Wd)winder 


Tests 


www.webwinder.com/auiz/auiz MC Refhtml 



Development of course content is not trivial, and teachers rarely develop course content 
without support from others. They seem to use content developed by others, collaborate, or 
work in teams. Some teachers participate in development courses, have support by web- 
designers, or support staff. 

Reusability and sharing of content could be usefial. Export features, archive functions, and 
standards could make this easier. Several comments indicate that the systems are perceived as 
flexible and open to differing didactic possibilities. But a few comments indicate that some 
systems could be perceived as inflexible with regard to didactic possibilities. To make course 
development easier or to increase productivity, some institutions provide or require that the 
content must adhere to special structures or templates. However, institutions that do not use 
the commercial systems maintain that their systems are especially designed and used to support 
their chosen didactics. 

In some cases, individual teachers design the content. In other cases, the teachers do not 
develop the course content themselves. But, it seems to be common that teachers collaborate 
or work together with other specialists to develop course content. Some claim that their 
system is easy to use by teachers and course developers. But obviously some support and 
training for teachers is usefial. This could include contact with experienced tutors, teacher 
training, dedicated discussion forums, and support services. 

Generally the interviewees confirm that the LMS systems support a wide range of media types. 
Most institutions seem to apply some multimedia. But, several interviewees are cautious about 
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too much use of multimedia due to bandwidth limitations or lack of development tools. And 
other institutions seem to be even more cautious about including online video. 

Assignment and assessment is a complex topic that should be applied in a pedagogic 
framework. The most frequent mentioned type of online test is multiple-choice questions. 
Among the other types of assignments and assessments mentioned are interactive 
assignments, quizzes, portfolio assessment, and surveys. Some of the interviewees are 
skeptic to computer-based assessment since it does not correspond to their pedagogical 
model or teaching tradition. Some of the systems have no built in tools for assignments or 
assessment. However, there are external tools that could be used to design assignments. 
The interviewees especially mentioned Autotest and Webwinder. WebCT seems to provide 
a range of useful tools. LUVIT, PingPong, Saba, and SSVN2000 also seem to provide 
useful tools for assignment and assessment. 



Student Support Tools 

There are user groups, such as sales people that work together in one company, that do not 
ask for online communication. But, the LMS systems seem to provide a lot of opportunities for 
student interaction through e-mail, distribution lists, discussion forums, chatting, bulletin 
boards, whiteboards etc. Discussion forums seem to be the most important tool for group 
communication. The institutions often organize a number of forums for various user groups 
and purposes. But, use of e-niail is also important, especially for institution with individual 
course progression. Chatting requires discipline and could be improved technically, but it has 
several interesting applications such as for example to reduce costs related to international 
communication. Among the other features that were mentioned for improving student 
interaction were tracking, integrated FAQ-services, personal presentations, pictures, and class 
lists. Finally, some comments reflected on students’ opportunities to interact with the course 
material, the environment, and other students. 



Many, but not all LMS systems, offer tools for both synchronous and asynchronous 
communication. Some institutions add external communication tools to offer additional 
communication services such as chatting, audio chatting, audio conferencing, and sharing of 
documents. Asynchronous communication via discussion forums and e-mail seem to be the 
preferred services for student-to-student communication. Several of the systems seem to 
provide some form of chatting as a tool for synchronous communication. The experiences with 
chatting are varying. It could be useful for social purposes and for informal communication in 
smaller groups. But several institutions limit the use of chatting since it is inflexible in time. 
Chatting may however be useful for formal project meetings. 



Technical and administrative support is a challenge that requires more or less resources. Both 
the numbers of support staff and support hours differ. Support is not available 24 hours a day, 
but some institutions offer support after normal office hours. The communication with tutors is 
handled both via e-mail and discussion forums. One institution claims that it guarantees 
students response from tutors in less than 48 hours. 

In general, the systems seem to have limited use of library resources. The most common 
services are links to existing internal and external Internet resources. Some institutions provide 
special services such as online journals, articles, library services, and bookshops for their online 
students. 



Some comments indicate that the LMS systems need improved functionality with regard to 
feedback on work assignments. And tutors seem to be pivotal with regard to feedback on work 
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and assignments. It is interesting to observe that some systems provide special mailboxes for 
submission of assignments, opportunities for online registration and presentation of grades, and 
real-time accounts for individual teacher remuneration. 

Tutor Support Tools 

Some institutions do not express a need for automatic tracking of student progression since 
their performance is measured by tutors or work competence. Tracking of students could, 
however, be useful for administrators, tutors, and students. But, one should be aware that 
some students are apprehensive about being monitored. Most systems seem to have some 
tracking of what students have done and when they did it. This may include which assignments 
they have completed, which web pages they have opened, and which tests they have taken. 
WebCT seems to provide tracking services that the users are satisfied with. They include 
tracking of quiz scores, log ins, pages accessed, and comments written. A few interviewees 
focus on the systems’ ability to provide advanced administrative reports on enrollments, course 
activity etc.. The self-developed SSVN2000 tracking system is especially interesting since it 
provides excellent tracking of students with individual progress plans. 

Group management includes tasks such as entering information about students, classes and the 
services they should have access to. This is easy to handle with few students, but could be 
much work in a large-scale system. Many systems allow teachers to form groups and establish 
discussion forums. Other systems rely more on system administrators for group management. 
In some systems, students may establish services such as chatting sessions and group calendars. 

Some comments express lack of functionality and tools for grading and examination results. 
Other comments indicate that there are many options and tools available for online assignments 
and that course designers should utilize the special pedagogical opportunities that are available 
online. 

Some interviewees focus on the ability for teachers or students to follow the students’ activity 
and progress. Other interviewees describe the administrative challenges of tracking students 
with individual progress plans. 

Some comments express a need for improved administrative systems between tutor and 
institution. The contact between tutors and the institution could be supported by special 
contracts, separate discussion forums, support services, face-to-face seminars, and training. 

Administration 

Some institutions have no need for integration between the LMS system and the economy 
system because they do not charge any tuition fees. WebCT does not seem to offer any support 
for payment of fees. NKI has developed the SESAM LMS system, which is fully integrated 
with the economy system. Most of the institutions have separate economy systems with little 
integration with the LMS. Some, do however, express ambitions and needs for such 
integration. Online payment via credit cards and special counters for calculation of variable 
fees could be implemented. NKN has special needs since it also handle online enrolment to 
face-to-face courses. 

Most of the institutions provide students with individual passwords. There are, however, 
examples of simple solutions in which many students share one password and advanced 
solutions that allow individual students to have just one password to all university systems. 
Most institutions seem to be pleased with their password systems. But there are diverging 
opinions on the workload generated by students who forget their passwords. User management 



could represent a heavy workload, which could be organized and automated in several ways. It 
is necessary to have a strategy:, for terminating the passwords. Distribution of virus could be 
reduced if the discussion forums are designed properly and encryption could be used to 
provide secure connections. 



The interviews indicate that the LMS systems are not especially successful for storing and 
retrieving student records. One institution has established a project to integrate the LMS with 
the student administrative systems; another sees this work as a major challenge. Other 
institutions have developed septate databases for information that is not handled by the LMS. 
Several institutions comment pn the needs and efforts to integrate the LMS systems with 
national student management systems in Sweden (LADOK) and Denmark (STADS). The users 
of SESAM, CourseKeeper, and Saba claim to have LMS systems that are well integrated with 
the student records databases. 



Some systems do not include examination and certification records. Other systems provide 
online information about grades. Several of the interviewees are concerned about the 
opportunities and challenges regarding integration with the administrative system that records 
the student grades. One interviewee from Denmark was concerned about privacy issues and 
how much information the systems should handle with regard to examination and certification 
records. Finally, one of the interviewees stated that exams should be larger and more project 
oriented to become more supportive of online education. 



Some of the interviewees provide positive statements about the facilities for administration of 
courses, classes and tutors. Other statements are more negative. It is especially interesting to 
observe that several systems seem to lack facilities and services on the level above individual 
courses. i 

Technology 

There are some free LMS systems that follow the open source policy, and the two users of 
open source systems were positive to open standards. There seems to be three categories of 
server solutions, and all seem to work well. In the first category, the institutions have access to 
commercial service providers that host the LMS. In the second category, the institutions host 
the LMS for internal use. And in the third category, the institutions host the LMS for internal 
use and as a service for other institutions. The institutions that have access to a service 
providers that hosts the LMS seem to be positive to the solution, but they experience some 
problems with limited access. Several institutions have chosen to host the LMS internally. They 
are typically either the institutions that have self-developed systems or larger institutions with 
high internal ICT competence that can operate commercial LMS systems locally. The users of 
the commercial systems claim that the systems are stable and reliable. The users of self- 
developed systems also experience few problems. Virus attacks and firewalls, however, are 
mentioned as serious problems. A few institution that have self-developed systems hosts the 
LMS for internal use and as a service for other institutions. One benefit of this is cost sharing. 



FirstClass depends more on client software than other LMS systems. Other comments point 
out that there is no need for any special client software. But, problems with firewalls could be 
difficult to solve. The institutions seem to differ on how much they rely on necessary client 
software for special courses. Several institutions seem to minimize the need for additional 
client software. But some special courses and subjects, for example about statistics, require 
additional client software. And other systems and courses rely with more or less success on 
additional client software such as special plugins or Microsoft Office products. 



119 



K Aa of Qn3iBfeliiiucatl(»t aiKl LeamiaaMai^eitiani. itt th& , 



LMS systems could be perceived as flexible, since both course content and system services 
could be updated regularly. Technology and templates limit the flexibility of the didactics, but 
it may increase the productmty. The interviewees are aware of the e-leaming standards 
specifications, and several claim that their system follow the standards. But few claim that the 
standards are important to their institution. 

The interviewees talk about LMS systems as large-scale systems capable of handling thousands 
of users. The interviewees are confident that the systems can handle a large number of users 
without special technological problems. The interviewees did not seem to be concerned about 
how the systems technically cOuld organize the administration of large numbers of students, 
courses, and tutors. One mentioned, though, that large-scale operation could impose some 
pedagogical challenges. 

Some comments express the fact that students have all kinds of connections to the Internet 
ranging from low speed modems to broadband access. But the speed of the LMS system does 
not seem to be any problem. The bottleneck seems to be the network bandwidth and local 
lines. To handle this, the institutions adapt their bandwidth requirements to the users’ 
equipment. Due to the bandwidth limitations, several of the institutions limit their use of high 
bandwidth content. 

Economic Issues 

Some of the interviewees view the cost of the LMS as confidential information. Others say that 
it is hard to estimate or that they don’t know it. The main costs reported on self-developed 
systems are related to a few positions in a development team and to server hardware and 
software. The costs and pricing structure for the commercial systems vary from system to 
system. This could make it difficult to compare the real prices. The costs mentioned range 
from under 5000 euros to over 100000 euros per year. None of the interviewees expressed 
sincere concern about high prices. 

One obvious advantage with self-developed systems is that the institutions pay no annual fees. 
The commercial systems have various pricing structures and prices for annual fees. The fees 
could depend on the number of user licenses, the number of years the contract is signed for, or 
just for actual upgrades. 

The Swedish and Finish institutions report that they have no tuition fees. Institutions in 
Denmark, Norway, and Iceland do report that they charge tuition fees, although many of them 
also receive additional funding from the state. The research did not find any examples of 
institutions that had implemented online invoicing. 

As the number of users in an LMS system grows, it seems necessary to divide the management 
and maintenance responsibilities among a number of people. It seems like the interviewees 
have vague knowledge about the systems’ maintenance and operation costs. The issue is 
perceived as complex and hard to estimate. The costs could be funded externally and it seems 
to include part-time work for from one to six internal people, but it could also constitute a fee 
per student to the ICT-department. 

It seems like the interviewees have little knowledge about how much time and money that is 
spent on trai nin g staff and students to use the LMS systems. The external costs could be low, 
since tr^ning primarily seems to be handled by internal staff. 
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Overall Evaluation 

There were several positive comments about the Nordic commercial systems. The systems 
were characterized as immediate, flexible, and open. The interviewees focused on their ability 
to save time and increase quality of learning, as well as their strong communicational and 
statistical features. It is especiedly interesting to observe that the interviewees value systems 
that support Nordic pedagogic^ traditions and national, academic user groups. The negative 
comments about the Nordic commercial systems were related to incompatibility with other 
platforms and products, limited student privileges, and the provider’s uncertain financial 
situation. 



The comments on overall evaluation on WebCT were predominantly positive. The interviewees 
used phrases such as: a good choice, offers all basic tools, works reasonably well, moderately 
cost effective, and easy to use! But, it could be hard to get support for local adaptations in 
Iceland. For example, WebCT version 3 was not available in an Icelandic version. 

The positive comments made about other commercial systems used terms such as simple, 
flexible, well working, many features, very satisfied, and provides most of the functionality we 
need. The most noteworthy negative comment about the other commercial systems was related 
to FirstClass’ use of client software. 



The users of self-developed systems are also satisfied with their systems. Some of the reasons 
for their choice are that they pbrceive the commercial systems as expensive and complex and 
that they can develop their systems to support their special needs. Among the advantages that 
were mentioned were the ability to handle continuous enrollment, cost effectiveness, and 
integration with student administrative systems and economy systems. 



Features in Future LMS Systems 

LMS systems could have reached a point were user-fnendliness, cost effectiveness, and 
integration with other systems is more important than new features. Some interviewees want to 
integrate the LMS with existing systems and other services such as student management 
systems, marketing catalogues, online payment, tracking of textbook shipments, registration of 
examinations, and multimedia tools. Others would like to have more flexible systems and tools. 
Several would like more use of multimedia, especially with regard to audio and video services. 
Some topics and languages need better representation of characters, symbols and user- 
interfaces. Other features the interviewees would like to see in the future were alternative ways 
to organize and visualize the learning process, better tools for synchronous communication, 
better ways to personalize design elements, and more national and international collaboration. 



Important Findings! 

The analysis resulted in a number of important findings that are listed in the following and 
sorted according to Nordic issues, integration issues, economic issues, and issues of special 
interest to providers of LMS systems. 



The following findings are especially related to Nordic issues: 

I. LMS systems seem to be widely used in Nordic higher, further, and continuing 
education. It is not easy to find such Nordic institutions without experiences with LMS 
systems. 

II. The analysis indicates that there is a clear trend towards large-scale online education in 
the Nordic countries. It shows that 12 of the 20 institutions offer at least 50 online 
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courses. According to ail 998-99 analysis, (Paulsen 2000) only 3 of 22 surveyed Nordic 
institutions offered more than 50 online courses three years ago. Further, the 
interviewees talk about LMS systems as large-scale systems capable of handling 
thousands of users. 

in. The Nordic universities have standardized on a few national student management 
systems. The systems are LADOK (Sweden), MSTAS (Norway), FS (Norway), 
STADS (Denmark), INNA (Iceland) and to some extent Oodi (Finland). 

IV. Nordic institutions seem to prefer LMS-systems developed in the Nordic countries. 
Among the 25 different LMS systems that were identified in the analysis, 16 were of 
Nordic origin. All other systems were of American, Canadian, or Irish origin. 

V. The research indicates that ClassFronter, WebCT, FirstClass, and BlackBoard seem to 
be the most used LMS systems in the Nordic countries. 

VI. The interviewees are aware of the e-leaming standards, and several claim that their 
systems follow the standards. But few state that the standards are important to their 
institution, and e-leaming standards do not seem to have had much impact on online 
education in the Nordic countries. 

Other important findings are related to the increasing need for integration between LMS 
systems and other online education systems; 

VII. LMS systems need to be integrated with a number of other systems in an organization 
that mms at providing efficient, large-scale, online education. 

Vin. The integration between the LMS systems and the student administrative systems 
seems to be relatively poor. 

IX. The integration between the LMS-systems and the economy systems seems to be very 
poor. 

X. Several of the interviewees are concerned about the opportunities and challenges 
regarding integration with the administrative system that records the student grades. 

Cost effectiveness becomes more important as the institutions become large-scale providers of 
online education, and the following findings are related to economic issues; 

XI. The costs and pricing stmcture for the commercial systems vary from system to 
system. This could make it difficult to compare the real costs. 

Xn. The interviewees have vague knowledge about the systems’ maintenance and 
operation costs. The issue is perceived as complex and hard to estimate. Further, it 
seems like they have little knowledge about how much time and money that is spent on 
trmning staff and students to use the LMS systems. 

Finally, these findings should be of special interest to providers of LMS systems who want to 
compete in the future market; 

XrH. The institutions do not seem to be especially loyal to, or dependent on, one provider 
of LMS system. The majority of the institutions had changed system, planned to change 
system, or operated secondary systems. 

XIV. LMS systems could have reached a point were user-fiiendliness, cost effectiveness, 
and integration with other systems is more important than new features. 

XV. The open source strategy may have an impact on the future LMS market. 

XVI. LMS systems are usually not used for development of course content. A broad range 
of external tools is used to develop the content before it is published in the LMS 
system. 
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XVn. It is especially interesting to observe that several systems seem to lack facilities and 
services on the level above individual courses. 

XVni. Several institutions; prefer self-developed systems. They perceive the commercial 
systems as expensive and complex and want to develop the systems to support their 
special needs. They wanted cost effective systems with the ability to handle continuous 
enrollment and integration with student administrative systems and economy systems. 
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The ABCs of E-Leaming 

The use of Learning Management Systems in the Czech Republic and 

Slovakia 



by 
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Abstract 

This article presents outcomes from an analysis on online education and 
Learning Management Systems in the Czech Republic and Slovakia. 14 In- 
depth interviews have been conducted with public as well as private 
institutions. 



The Candidate Countries (CC13) are fully aware that they must make greater 

9 efforts than the EU Member States (EU15) if they are to be a part of the 
future, integrated European Information Society. Although most countries 
have made significant progress since the reform process began in the early 
1990's, there are still many areas where the current situation is still far 
behind that of most EU Member States. For example the average access/use 
of a computer is 45% in EU15 and only 25% in CC13 or access/use of 
Internet represents 30% in EU15 and 18% in CC13. 

The implementation of the eEurope+ 2003 Action plan is based on a simple 
strategy, namely on the one hand it is based on a common set of actions 
(clearly identifiable, concrete actions and target dates) contained in 
elaborate national eStrategy Plans in each of the Candidate Countries, while 
on the other hand the actions are directly linked to eEurope 2002 and to the 
next phase eEurope 2005 in order to ensure a broader European relevance 
and to avoid a digital divide with the EU. 

^ Czech and Slovak e-learning and the use of LMSs is finding its stable position 
and muddling through financial obstacles, lack of political will and general 
mistrustfulness. Nevertheless the 14 interviewed institutions have 

experience with 10 LMSs. The most used system is Czech TUTOR2000 
(currently by 5 institutions). 3 interviewees have developed their own 
systems and the last 6 represent American commercially available LMSs 
(BlackBoard, Click2learn, GLN {Cisco}, Intralearn, LearningSpace and 
WebCT). 



9 from 14 interviewed institutions have not been using LMS longer then one 
year. That shows the general very early start of online education in these 
countries. In spite of that there is already a quite high number of students 
included in the courses (up to 4005). This market is noticing a high 
acceleration of this development and it might be predicted into future as 
well. 
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This paper refers to online education and LMSs in the Czech Republic and 
Slovakia. 



/Country 


Language 


Geographtraip|ei| 


/ •Populatlori/'/y:! 


The Czech Republic 


Czech 


78 864 


10.287^'* 


Slovakia 


Slovak 


49 035 


5.389^^ 


Total 


i 


127 899 


15.676 



The survey ran in two stages: 



I. In the first stage research on institutions, which are active in the e- 
learning field and possibly use IMS, had been conducted. Many times 
there was the chancel to speak to the responsible people from various 
institutional bodies including the main universities, private companies 
providing online education as well as LMS providers/vendors, who 
brought an objective picture of the country situation into the study. 
With their advice a high number of potential respondents have been 
contacted and they all have received a translated interview guide in 
Slovak^®. There were seven main parts identified in the Web-edu 
project questionnaire: 



1. Course development tools 
^ 2. Student support tools 

3. Tutor support tools 

4. Administration 

5. Technology 

6. Price aspects 

7. Conclusion: 

a) Overall evaluation 

b) Features in future LMSs 

II. In the second phase the institutions had been interviewed during the 
time period March-May 2002, the questionnaires were collected and 
translated back into English. Interviews with 14 institutions, 10 from 
the Czech Republic and 4 from Slovakia, create the basis for this 
analysis. Among interviewees are: 

Academic institutions : universities, colleges, faculties, 
units belonging to universities; 



Source: Czech Statistical Office, figure from March 2002, http://www.czso.cz/eng/angl.htm 
Source: Slovak Demographic Research Centre, figure from August 2002, 
http://www.infostat.sk/vdc/expiorer/pophoden.htm 

Remark: Czech and Slovak language is very similar and there is no problem e.g. for a Slovak students to 
study in the Czech Republic or vice-versa. 
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other institutions : private companies using LMS for 
internal purposes, private companies providing online 
courses as their core business, civic association 
providing education. 



This analysis could not have been carried out without the kind assistance and 
help of many people and institutions. We would like to express our thanks to 
them all for their support, provision of their experience and expertise. 



Background 

The EU Candidate Countries are faced with enormous challenges in their 
attempt to catch up the EU development in several spheres, including the 
development of a knowledge-based economy. The process of transformation 
from a planned to an open market economy is taking place at the same time 
as accession to the European Union. The unequal starting points of the 
^ Member States and Candidate Countries have been already recognised by 
both sides and are described in the following sections. 

Technological setting 

According to the survey Candidate Countries Eurobarometer 2001^^ (one of 
the tools of European Commission's Public Opinion Analysis 



Sample Specifications 

Between the 1st and the 28th of October 2001, The Gallup Organisation Hungary carried out wave 2001.1 of the 
Candidate Countries Eurobarometer (CCE), at the request of the EUROPEAN COMMISSION, Dlrectorate^General Press and 
Communication, Public Opinion Analysis. Its methodology Is almost Identical to that of the Standard Eurobarometer being 
conducted from 1973 In the EU through approx. 1000 tace-to-tace Interviews per Member State. Therefore this was the first 
time that the results could be compared. CCE replaces the Central and Eastern Eurobarometer which was carried from 1990 to 
1998. 



The Candidate Countries Eurobarometer 2001.1 covers citizens of each of the countries that are applying for 
European Union membership aged 15 and over, with the exception of Estonia and Cyprus. In Estonia, the survey 
covered permanent residents aged 15 and over. In Cyprus, the survey only covers citizens living on the southern 
part of the island. The basic sample design applied in all Candidate Countries is a multistage, random (probability) 
one. In each country, a number of sampling points were drawn with probability proportional to population size (for a 
total coverage of the country) and to population density. 

For doing so, the points were drawn systematically from each of the "administrative regional units", after 
straUfication by individual unit and type of area. They thus represent the whole territory of the Candidate 
Countries Region according to the EUROSTAT NUTS 2 (or equivalent) and according to the distribution of the resident 
population of the respective nationalities in terms of metropolitan, urban and rural areas. In each of the selected 
sampling points, a starting address was drawn, at random. Further addresses were selected as every Nth address by 
standard random route procedures, frorn the initial address. In each household, the respondent was drawn, at 
random. All interviews were face-to-face in people's home and in the appropriate national language. In countries with 
significant minorities the respondents had a chance to respond in their mother tongue (in Estonia, Latvia and 
Lithuania in Russian, and in Romania in Hungarian). 



Data used in Candidate Countries Eurobarometer 2001 



Countiy 


BG 


CY 


cz 


EE 


HU 


LV 


LT 


M 

T 


PL 


RO 


SK 


SI 


TR 


Total 


# of 

interviews 


1000 


500 


1124 


100 

0 


1016 


100 

4 


100 

6 


500 


1001 


1010 


1044 


100 

0 


1000 


12077 


Population x 
000 


8,487 


663 


10,229 


1,44 

6 


10,198 


2,43 

9 


3,70 

1 


379 


38,666 


22,546 


5,391 


1,98 

6 


56,473 


162,60 

4 
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http://euroDa.eu. int/comm/public opinion/cceb en.htm ^. which was released 
by European Commission (from the fieldwork of October 2001) in March 
2002, following results can be seen. 

In table 2 the selection of the most relevant kinds of information technologies 
is stated for all 13 Applicant Countries. Except introduced technology types 
the survey provides information also about the access or use of a: video 
player or recorder; fax; satellite dish; television fitted with teletext; CD-ROM 
or CDI-reader; or none of these as you will see in the figure i. 

Table 2. Access to modern information technology (°/oby country)^^ 



Question in the survey: Do you have access or do you use... ? 



Country 


A computer 


The Internet, 
the Worl^ I 
Wide Web 


A modem 


A mobile 
phone 


iThe Czech ^ ^ -i! 
i Republic 


AA 


31 


■ ' .17 




Slovakia 


i ■ 29 


1 is 


; "iO 


! ' 43 


Bulgaria 


14 


11 


7 


14 


Cyprus 


49 


34 


23 


58 


Estonia 


35 


32 


19 


56 


Hungary 


29 


16 


12 


47 


Latvia 


35 


24 


11 


45 


Lithuania 


26 


18 


9 


41 


Malta 


38 


28 


22 


57 


Poland 


33 


21 


16 


39 


Romania 


18 


10 


6 


26 


Slovenia 


53 


39 


36 


76 


Turkey 


18 


16 


8 


51 


;AVARAGECC13 


25 


: 






43 


iAVARAGS 


L- '45 










Ihict^^ieasur^ in 
SSB55.1 



As it is seen from the table 2 the most advanced country from the CC13 In 
"Information technology" is Slovenia, where more than the half of the 
population works with the PC and almost 40% have the access to the 
Internet. In these two mentioned criteria then only Cyprus exceeds the EU 
average values and the Czech Republic is very close to these figures. 
Slovakia stands worse than Estonia, Latvia and Malta as well as Poland is a 
bit ahead. We can say that Slovakia is more or less on the average of CC13. 
However we have to consider that these indicators are deformed by a high number of 
inhabitants of less advanced countries as Romania and Turkey, then Slovakia actually would 
be placed at the end positions of the ranking. 



For each country a comparison between the sample and the universe was carried out. The 
Universe description was derived from population data from national statistics. For all 
Candidate Countries a weighting procedure, using marginal and intercellular weighting, was 
carried out, based on this Universe description. As such in all countries, gender, age, region 
NUTS 2, settlement size, household size, and education level were introduced in the 
iteration procedure. For international weighting (i.e. CC averages), Gallup applies the official 
population figures as provided by national statistics. See above in the Table. 



^ Source: Candidate Countries Eurobarometer 2001.1, October, 2001 
^ Source: Standard Eurobarometer (SEE) 55.1, Apr-May, 2001 
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In following figure 1 the comparison between average data of EU15 and 
CC13 is clearly showed. In this respect the Candidate Countries lags 
significantly behind the European Union. 



Figure 1. Access to Information Technology 

% access/ or use “ 



None of these (spontaneous) 
A fax 
A modem 

A decoder for pay-TV programmes 
The Internet, World Wide Web 
A CD-ROM or CDI-reader 
A computer 

A satellite dish to pick up TV programme 
A video recorder 
A mobile phone ** 
A television fitted with teletext 




□ CC13 HEU15’*' 



100 



Ongoing initiatives 



In order to eliminate this huge gap between EU Member States and New 
Incomers special initiatives have been launched. 

At the European Council held in Lisbon on 23-24 March 2000, the Heads of 
Government and State of the EU-15 set the ambitious goal for Europe for the 
^ next decade to become "the most competitive and dynamic knowledge-based 
economy in the world". It recognised the urgent need for Europe to quickly 
exploit the opportunities of the knowledge-based economy and in particular 
the Internet. To achieve this objective a comprehensive eEurope Action 
Plan / eEurope 2002 

('httD://www.euroDa.eu.int/information society/eeurooe/action olan/pdf/acti 
onolan en.odfi was developed and adopted by the European Commission in 
May 2000 and finally launched in Feira on the 19-20 June 2000. 



At the European Ministerial Conference held in Warsaw on 11-12 May 2000, 
Central and Eastern European Countries recognised the strategic goal set by 
the EU-15 in Lisbon and agreed to embrace the challenge set by the EU-15 
with eEurope and decided to launch an "eEurope-Hke Action Plan" / 
eEurope+ 2003 (httD://www.vus.sk/is/doc/eEuroDe%2Ben2001.pdn by and 
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Source: Candidate Countries Eurobarometer 2001.1, October, 2001 
^Source: Standard Eurobarometer 55.1, Apr-May, 2001 
^^Mobile phone access was not measured in Standard Eurobarometer 55.1 
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for the Candidate Countries as a compliment to the EU political commitments 
in order to try and broaden the base for achieving the ambitious above 
mentioned goal. In February 2001, the European Commission invited Cyprus, 
Malta and Turkey to join the other Candidate Countries in defining this 
common Action Plan. 

The eEurope+ Action Plan / eEurope-¥ 2003, launched at the occasion of 
the Goteborg European Council (15-17.6.2001), mirrors the priority 
objectives and targets of eEurope and defines actions specific to the situation 
in the Candidate Countries. 

Actions in eEurope+ are clustered around the same three main objectives 
identified in eEurope (see Table 3) and the same indicators selected by the 
EU-15 are adopted for monitoring and benchmarking of the progress. 
However, the Candidate Countries recognise that, if the full benefits of the 
actions are to be achieved, a further acceleration in the effective 
implementation and functioning of the acquis communautaire in areas 
related to Information Society is required. This has resulted in the inclusion 
of an additional, new objective (zero objective and the other three are the 
same as in eEurope 2002), not previously found in eEurope, that aims to 
assist in putting in place the fundamental building blocks of the Information 
Society. Furthermore attainment of the eEurope+ objectives can be 
significantly enhanced and accelerated through cross-border and 
international collaboration. 
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Table 3. Objectives in eEurope + 2003^^ 



pl»Ge of 

f'' ' for the' 
So#ety , 


a) Accelerate the provision of affordable communication 
services for all 




b) Transpose and implement the acquis relevant to the 
Information Society 


; L A cheaipieir^ faster, s^'re xn'teraet 


a) Cheaper and faster Internet access 




b^ Faster Internet for researchers and students 




c) Secure networks and smart cards 


\ 2; tisvesting in n^pie and Millie 


a) European youth into the digital age 


it. ■ ^ ' 


b) Working in the knowledge-based economy 


12. ' ' 


c^ Participation for all in the knowledge-based economy 


: 3. Stimniate nse nf the internet 


a) Accelerating e-commerce 




b) Government online: electronic access to public 
services 




c) Health online 




d) European digital content for global networks 




e) Intelligent transport systems 




f) Environment on-line 



Outcomes 

Currently there was already a Progress Report on the implementation 
of the eEurope+ 2003 Action Plan in the Candidate Countries 

published rhttD://emcis.QOV.si/mid/emcis.nsf/V/K89BFB6D139731A05C 
1256BCAQ0444679/$file/ProQress report. odfi . which was presented 
during a conference in Ljubljana (Slovenia) on the 3'"'* and 4^^ June. 
Ministers and other representatives from the 13 Candidate Countries 
and several EU Member States met there on the invitation of the 
Slovenian Government and the European Commission. 



The results are mirrored by press releases from June 2002 about eEurope+ 
Action Plan discussed in Ljubljana : 



Presenting key figures from the report, Robert Verrue, who is Director 
General of the European Commission's Directorate General for the 
Information Society, underlined the importance of progress made by the 
candidates in the "building blocks", namely, legislation in this field. Quoting 
figures from the report, Verrue pointed out that the applicants have 
harmonised 80 percent of their telecommunications laws to EU legislation. 
Moreover, Verrue also pointed to the fact that the applicants are taking steps 
to improve basic access to communications, thus catching up with the 
member states in this field. The European Commission official thus pointed to 
the percentage of households with fixed telephone lines - which now stands 
at 77 percent in the applicants and 86 percent in the members. However, 
Verrue also reminded of the need to do away with obstacles that obstruct 
further progress in the applicants, among the most burning being the cost of 



Source: eEurope+ httD://www.vus.sk/is/doc/eEuroDe°/o2Ben2001^Bdf 
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services and computers. He also pointed to the great disparity in progress 
between the various applicant countries and even between different regions 
of one country - whereas an urban area may have 100 percent coverage with 
fixed iineSf a rural area may only reach 30 to 40 percent 
Some aspects of the development of our two countries in this analysis were 
also stressed and described in more details in the Progress Report: 

In the chapter of "Capabilities and Skills": 

The Slovakian INFOVEK Project aims to provide Internet access to 2,500 
primary and 800 secondary] schools by the end of 2005. Almost 20% of the 
schools had been connected to the Internet at the end of 2001 and this 
proportion should increase to 35% by the end of 2002. In addition to the 
Internet connections teachers are being trained to use ICT and integrate it 
into the teaching and learning process using multimedia materials and digital 
content. 

The Czech Republic is currently implementing the first period of information 
educational policy called internet for Schools'. It aims to make infrastructure 
available for all schools by 2002, improve the access by 2003 and ensure ^ 
that high quality training is available at primary and secondary schools by 
2005. The objective of the project is to ensure a high quality ICT literacy for 
primary and secondary school leavers with support of improved infrastructure 
and software. In total, 6,200 schools should be put online in the framework 
of this 250 M Euro project by 2005. 

As well as in the section "Research Networks": 

CESNET (Czech National Research and Education Network), established in 
1996 by all universities of the Czech Republic and the Czech Academy of 
Sciences, has recently upgraded the CESNET2 connection to the Internet 
from 155 Mbs to 622 Mbs. CESNET's main goals are operation and 
development of the Czech NREN, research and development of advanced 
network technologies and applications and increased public awareness about 
advanced networking matters. ^ 

The Slovakian academic network (SANET) has been substantially improved 
and the bandwidth of the backbone network has been upgraded from 4Mbps 
to 1 Gbps. AH the major cities will be connected by the end of 2003. SANET 
provides access to Internet for all universities, research institutions, scientific 
libraries and some schools and museums. 
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Obstacles and problems 



The Progress Report shows that the Information Society is already present in 

EU Candidate Countries, on the other hand there are still obstacles and 

problems, which can be surrimed up as follows: 

• Fixed line networks are known to have substantial levels of faults per line 
(particularly where analogue switches, old loops, and shared lines are still 
used) and there is poor coverage in rural areas; 

• Despite high penetration rates, it can not be assumed that the current 
generation of mobile networks are usable for access to Internet although 
the current upgrading technologies (e.g. GPRS) offer potential in 
anticipation of 3G services (UMTS); 

• Cable TV availability has not yet been exploited at any scale for use to 
access Internet. This may be a policy issue that needs to be addressed; 

• Alternative access technologies such as wireless local loop, DSL, and 
digital TV are not yet widely deployed, if at all; 

• Both the costs of Internet access as well as the cost of purchasing a 
personal computer representing a significant portion of the net income 
appear to act as blocking factors in household penetration rates for 
Internet access; 

• With a few exceptions, there is still a low penetration of computers in 
schools. In addition, there is substantial divergence between the countries 
for all three levels (primary, secondary, and tertiary). It seems that about 
half of the computers in the schools are connected to Internet, in some 
countries with a very high-speed connection via national research 
networks. Most of the countries have ambitious programmes that aim to 
connect schools and provide computer facilities; 

• Pubiic access points remain a very important means of Internet access for 
the population at large; 

• Overall, it can be said that there are considerable divergences between 
the candidate countries in absolute terms. Some seemingly do better than 
many of the EU Member States, at least in the areas in which there is 
data, but many still have considerable catching-up to do. 



Which way to go? 

During the last few years, an important issue has been that of "materialising 
the political will". In other words, how to get from policy to practice? Policy 
driven development versus development driven policy? It is probable that a 
development driven policy may work better in the candidate countries as the 
implementation mechanisms are not yet operating at an optimal level and 
public administration reforms are still in process. 

With many of the candidate countries currently experiencing the first, rather 
difficult results of the liberalisation of their telecommunications markets, 
attention now needs to be given to effective enforcement of the pro- 
competitive regulatory framework in order to bring prices down and 

Maria MicincQV&:Thie ABCs of 
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penetration up. Special attention will need to be given to the preparations for 
the Implementation of the !new EU regulatory package for communications 
services, recently adopted by Council and European Parliament. 

Furthermore, important issues for the next phase of the eEurope+ action 
plan are: the completion of the implementation of the EU acquis relevant to 
the Information Society, in particular in relation to eCommerce as a pre- 
condition in creating trust and confidence in the use of Internet-based 
transactions; the introduction of alternative Internet access technologies; the 
provision of computers to schools and their connection to Internet, 
accompanied by appropriate curricula and training of teachers; increasing the 
number of public access points to ensure greater participation for all; and the 
further development of eGovernment services and of local content. 

All these ideas are huge tasks for the Candidate Countries; they will go on in 
their clear and tangible commitment to progress the implementation of the 
Information Society. This cbmmitments is more than ambitious considering 
the fact that they have focused on a common key date - 2003 - by which 
they aim to meet the eEurope+ targets and at the same time the EU Member 
States have already adopted a continuation of eEurope 2002 a new Action 
Plan eEurope 2005 which covers years 2003 - 2005 during the Seville 
European Council 21 and 22 June 2002 and this follow-up document was also 
already recognised in its core priorities at the above mentioned conference 
in Ljubljana in June of this year by all participating countries that means by 
Applicant Countries as well. 

(http://www.europa.eu.int/information society/eeurooe/news librarv/docum 
ents/eeuroDe2005/eeuroDe2005 en.pdH 

Institutions and their LMSs 

Ten Czech and four Slovak interviewees will from now on guide us through 
following pages. Table 4 contains their basic data. According to the division 
stated on the first page there are 9 academic bodies (7 from the Czech 
Republic and 2 from Slovakia) and 5 so called other institutions (3 from the 
Czech Rep. and 2 from Slovakia). 



BEST COPY AVAILABLE 



j , Maria of 

tlie tfse (rfijeaaiaig Madia^inent ia the CStech RepuMic aisd 



Table 4. Institutions sorted by Country 



Institution ' 


URL of 
Institution 


^ . Country : : 


;:j Typo of fiii^tutiph f. 


Ifaiversity of Hradec Kralove, Faculty of 
Informatics and Management (UHK-FIM) 
Univerzita Hradec Kralove, Fakulta 
infbrmatiky a managementu 


www.uhk.cz 


Czech 

Republic 


Public College 


VSB - Technical University of 
Ostrava (VSB - TUO) 

Vysok^ §kola b^fisk^-Technick^ 
univerzita Ostrava 


www.cs.vsb.cz 


Czech 

Republic 


State University 


The Faculty of Economics, VSB - 
Technical University of Ostrava 
(FE) 

Ekonomick^ fakulta, Vysok^ skola 
birtsk^-Technick^ univerzita Ostrava 


www.ekf.vsb.cz 


Czech 

Republic 


College - University 


r/ie University of Ostrava 
Ostravsk^ univerzita 


www.osu.cz 


Czech 

Republic 


University 


^he School of Business 
P^dministration in Karvina, the 
Silesian University in Opava (SBA 
in Karvina) 

Obchodne podnikatelsk^ fakulta v 
Karvind, Slezsk^ univerzita v Opave 


WWW.ODf.SlU.cz 
httD://edu. 0Df.slu.cz 


Czech 

Republic 


Public College 


The Faculty of Science, The University of 
Ostrava (FS-UO) 

Prirodovedecka fakulta, Ostravsk^ 
univerzita 


www.osu.cz 


Czech 

Republic 


University 


Czech Technical University in Prague (CTU 
in Prague) 

£esk4 vysok6 uCenf technick4 v Praze 


www.cvut.cz 


Czech 

Republic 


Technical University 


The Czech-Swiss Institute 
Cesko-Svycarsky Institut 


www.vkacsi.cz 


Czech 

Republic 


Independent Institution 


The Czech Insurance Company p.l.c. 
£esk^ pojiSfovna a.s. 


WWW.CDOl.cz 


Czech 

Republic 


An Insurance Company 
Public limited company 


Czech Telecom p.l.c. 
Keskv Telecom a.s. 


www.ct.cz 


Czech 

Republic 


TELCO provider 
Public limited company 


P : • ; : -v; , 

fATNova) • • 


www.ainova.sk 


Slovakia 


Civic Asspciation 
Ihd^ndent rioh“ . 

gdvettimental Institution ' 


UmverstiyofTectotSdgymBr^ : 

(LCDB) - - • •: ; 

tofc^)iWsbredlsl«>dlStao^R$h^^ 
vzdefiv^ia pfi Slovenst^JTechhickej ' 
jLJhlvt^lte Bratislava- . ^ 


http.t/jTieixE 
elf. stifba Jsk/lsdv/ 

. . • . • .fj- .. 


.'Slovaida""".'i 


' 'Educational 'iniStl^ ' ' ' • " " 


TefeOom 


www'.teled^ ' 


'Slovakia 


Educational :i 

centra. y 




www.ute,sfc 


Slovakia 


university:- il; -.ji'l-iy 



I The Czech Republicl 
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Following table 5 is a sumrnery of the most important quantitative data on 
use of LMSs by institutions. Among the 10 identified LMSs, only 3 
(BlackBoard, Click2learn, LearningSpace) are listed as "other LMS". That 
means that currently they are not used as the main systems, they might 
have been applied in the past, or might be in use at the moment as 
secondary systems. Products by Macromedia and a toll Instructor 
(Click2learn) are not included in the study as LMSs. More transparent 
overview of LMSs is sated in the table 6. In the further analysis we will work 
only with the currently used systems. 

We can see that we deal with three major groups of LMS systems. The first 
one Is the system Tutor2000, which is in use by 5 institutions. The second 
group could be called other standard LMS comprising WebCT (2x), GLN, 
LearningSpace, Aspen, and Intralearn. The last one represents the In house 
developed systems used by three institutions. 

The number of online courses does not vary significantly. The highest 
numbers are 30 courses and 40 modules. 

The table shows bigger range in the number of online tutors (from 2 to 200) 
as well as in the number of online students (from 30 to 4005). For example, 
Tutor2000 is in use by all contacted institutions just for very short time yet. 
By all of them it is just their first year, in this respect the course length is 
not yet clearly specified. In spite of that there is quite a high number of 
students included in the courses already (up to 4005). 

The column of number of years in use indicates the early start of e-learning 
application in the Candidate Countries. There are only 5 institutions that 
have more than 1 year of experience. 

The typical course length is usually determined by the school terms at the 
universities, which is 14 weeks. By other institutions this figure varies from 
20 hours to 6 months. 



Table 6 shows the origins of the LMSs. There are 5 of American nationality 
currently used by 6 institutions. On the other hand there is one Czech 
system used by 5 respondents. 3 institutions have decided to develop their 
own system according to their requirements; they are Czech as well as 
Slovak origin. 
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Table 5. Data on use of LMSs by institutions sorted by LMS 



N^ejof :i;.- 
in^tititSory T:; ' . 






courses.. 


mmm- 1 

'Online 

Uitors 


'pdleintlli 


mmm 

iiniliSiiiii 


i|iicbotf|:l|j 


VSB - TUO 


GLN - Global 
Learning Network 


None 


The system is 
universal. At the 
moment there 
are about 8 
courses 
provided. 


2 per one 
course 


no 

limitations, 
but approx. 
80 students 


2 


6 months 


Czech 

Telecom 

D.I.C. 


Ingenium 6.1 = 
Aspen LMS today 
Click2leam 


None 


20 


more 
than 10 


15,000 


2 


3 days 
(FTF 

equivalent) 


AINova 


Is house 
de^^ped 


N<»e 


1 


12 


30 


Q 


&m(»ths 


Czech- 

Swiss 

Institute 


In house 
developed 


None 


40 modules 


20 


Currently 
about 100 
students 


2-3 


1 month 


|||||g||||i 


In house 
tTev^oped 


None 


2 


14-6+6 


30+30+300 


1 


•Arbitrary 


TeleDom 


IntraJearn 


BfackBoard 


30 


3Q 


ISO 




20 hours 


FS-UO 


LeamingSpace 


Click2Learn 


30 


20 


100 


5 


One term 
(14 weeks) 


Czech 

Insurance 

Company 


Tutor2000 


None 


Currently 
approx. 25 


Currently 

6-10 


4000 


1 


2 hours 


FE 


Tutor2000 


None 




200 

teachers 


4005 


6 

mont 

hs 




SBA in 
Karvina 


Tutor 

Instructor 


Only products 
by 

Macromedia 
(Director, 
Dreamweaver, 
Flash and 
Authorware) 


6 


8 


400 


Since 

Septe 

mber 

2001 




University 
of Ostrava 


Tutor2000 


Before 

LeamingSpace 


3 developed 
courses 


2 


50 that are 
signed and 
working, 
approx. 500 
students 


The 

first 

year 


One term 
(14 weeks) 


University 
of Zilina 


TutQr2Q00 


Ncme 




6 


A Jiceice for 
1000 
^udents 


Proba 

□per3 

dCHl 

for 5 i 
man!; i 
his 


It Is not 
Specified 


CTU in 
Prague 


WebCT 


Nonia 


Not limited 


Approx. 

15 


More than 
300 


2 


Support for 
traditional 
courses, 
which last 
one term 
(14 weeks) 


UHK-FIM 


at the moment 
WebCT 


Before 

LeamingSpace 
(for 2.5 years) 


8 at the 
moment, 15 in 
preparation 


20 


More than 
500 

(students in 
the courses) 


1 


One term 
(14 weeks) 



Status: March - May 2002 (time period of conducting the interviews with the institutions) 
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Table 6. LMS systems sorted by original nationality 



IMS sysfenis 


' Original 
National! 


Language 
Olf LMS 


URL of LMS 


Institutions 
currently using 
stated LMS os a 
main insfrunient 


Institutions 
usiftg other 
LMS 


BlackBoard 


American 




www.blackboard.com 




TeleDom 


Click2learn 
Xngenium 6.1 = 
Aspen LMS today 


American 


Englisih 

Czech 


httD://home.click21eam.coin/en/as 


Czech Telecom 


FS-UO 


oen/asoen Ims.aso 


www.click21eam.com 


GLN - Global 
Learning Network 
Cisco 


American 


English, 

German, 

French, 

Spanish, 

Hungarian 

/ 

Japanese 


httD.7/cisco.netacad.net 


VSB - TUO 


i 




Xntralearn 


American 


" : 


httD://teledom.ibcnGt.hu/ 


TeleDom 




httD://www.intraleam.com/ 


LearningSpace 

(Lotus-IBM) 


American 


English 


httD://www.lotus.com/t)roducts/lea 


FS - UO 


UHK - RM 


msoace.nsf/wdocs/homeoaee 


httD://domino.osu.C2/weboaee2.nsf 


WebCT 


Canadian 


English 

Czech 


www.webct.com 


UHK- FIM/CTU 
in Prague 




httD://oliva.uhk.cz/webct/Dublic/h 


ome.Dl (of University of Hradec 


Kralove, RM) 


In house 
developed 

Czech-Swiss 

Institute 


Czech 


Czech 




Czech-Swiss 

Institute 




Tutor2000 
(company Kontis) 


Czech 


Czech 


www.kontis.cz 


FE/ University of 
Ostrava/ SBA in 
Karvina/ Czech 
Insurance 
Company/ 
University of Zilina 




www.e-leam.cz 




Xn house 

developed 

AINova 


Slovak 


English 


httD://www.flexeman.sk 


AINova 


€ 




Xn house 

developed 

LCDE 


Slovak 


Slovak 


httD://hercules.kar.elf.stuba.sk/~tar 


LCDE 




erouD/snls/start/ 


httD://hercules.kar.elf.stuba.sk/tar/ 


oroiects 


htto://hercules.kar.elf.stuba.sk/tar/ 


ts/ 



Status: March - May 2002 (time period 



of conducting the interviews with the institutions) 
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Results from the interviews 



There were seven main parts identified in the Web-edu project 
questionnaire: 



1. Course development tools 

2. Student support tools 

3. Tutor support tools 

4. Administration 

5. Technology 

6. Price aspects 

7. Conclusion: 

a) Overall evaluation 

b) Features in future LMSs 

As already mentioned we deal with three major groups of LMS systems 
defined as: 

- Tutor2000 used by 5 from 14 institutions 

- Other standard LMSs : WebCT (2x), GLN, 
LearningSpace, Aspen, Intralearn 

- In house developed systems bv three 
institutions 

Each main part follows this classification. Considering the number and 
variety of systems in the second group, the description is done by means 
of tables and following indicators: 



- (+) "positive" impression 

- ; (-) "negative" impression 

- (+/-) "rather positive than negative" 

impression 

- (-/+) "rather negative than positive" 

impression 

- (?) "overall" 



The statements in these tables show the opinions of the interviewed 
institutions on the use; of their specific LMS according to the single 
questions. The indicators then reflect the sum of these individual 
opinions. The indicators on the bottom of each table express the quality 
of the certain LMS according to the given statements. The indicators on 
the side of the table show the general impression of all 5 standard 
systems within the single question category. There are cases by 
description of WebCT that the column is divided into two statements. The 
reason is that there are two institutions using this LMS and the separation 
Is always applied when their answers differ from each other. In other 
cases when they supplement each other or their impression is the same, 
it is placed in one square. 

The indicators are not included in the table of price aspects because it is 
a quantitative description. 



me 



' Maria MftaBcova: The ABCs of B^jeaming, 

The Use afteatning Maaagemest Systems iatbe C^edh Re|)hbHc ^ SlowaJda 



o Course developnient tools 

We were looking for answers on following questions: 

o Course creation. How satisfactory was the LMS for course 
creation? 

o structure and didactic fiexibiiity - openness. In the creation of 
course materials did the LMS permit didactic flexibility? Was the 
structure open to differing didactic possibilities? 

o Teacher user friendiiness. How easy was the LMS to use by 
teachers and course developers? 

o Support for graphics, audio and video, moving image. Did the 

LMS support the provision of graphical materials, moving images, 
audio and video in the course content? 

o Questioning, assessment, assignments. What provision was 

made by the LMS for student questioning and assessment and the ^ 
design of student assignments? 



Table 7. Software Tools Used for Course Creation 



Software tool 


URL • 


ToolBook Instructor (Click2learn) 


httD://home.click21eam.com/en/toolbook/index.asD 


Macromedia 


httD ://www.macromedia.com/ 


Flash 




Director 




Dreamweaver 




Authorware 





Tutor2000 

By all institutions using Tutor2000 came the same answer saying and 
differentiating the course creation from the LMS, LMS does not influence 
the course creation. Tutor2000 is a management tool not a content 
creation tool. For these purposes they apply other tools as stated in the 
Table 7, mainly ToolBook II Instructor, which is viewed as fully 
satisfactory. 

The system permits the didactic flexibility and the structure is open to 
different didactic elements like graphics, animation, simulation, video, 
audio and allows their interconnection. 

Generally the work with the system requires some training and the co- 
operation with creators is needed. 

Concerning questioning^ assessment, assignment there were following 
reactions: 




BEST 



COPY 



AVAILABLE 




"AH the tests (those generated automatically as well as those created 
manually) have the character of independent courses that means they are 
the same objects like usual courses within the LMS." 

"Student self-control, student assessments, various forms of testing, 
giving assignments are possible." 

A negative respond as well: 

'TutorZOOO does not have its own questioning system. With the help of 
standard communication AICC the system can obtain results of the tests, 
which were created in ToolBook. These results then can be processed by 
the system. Unfortunately our installation of TutorZOOO shows 
considerable defects and so this function does not work reliably (at the 
moment not at all ). " 



Other standard LMSs 



Table 8. Course development tools - Other standard LMSs 



i. Cout^e development tools 




WebCT 


GLN 


: intraleam 


Adped 


? 


ia 


Good for structure 
creation, worse with 
editing 

provisions(symbols, 
formulas), multimedia 
is not good, transfer of 
text not well solved 


More or less 
satisfactory 


Course 
materials 
developed 
outside LMS, 
implemented 
in format of 
HTML, XML, 
PDF 


Course 
created in 
advance by 
LMS provider 
(Cisco 
Systems) 


Easy usable Internet 
platform for creation 
of dynamic, 
interactive, 
measurable Internet 
products 


No course 
development 
tools. It is 
"open" LMS. 


/ 

+ 




More or less, general 
course overview, 
course tasks, time 
figures, overview of 
reached results are 
satisfactory 


Quite open; 
enables to 
use various 
didactic 
elements 


some tools 
are 

provided, 
but not 
possible to 
create 
another tool 


No creation 
of materials 
needed 


Material addition very 
flexible 


Yes, but 
limits in 
multi level 
course 
structure - 
not available 


+ 

/ 


1.3 


Not that easy, 
requalification needed 


Quite easy after short 
training or with knowledge 
at least on level ECDL Start 


Easy, 

intuitive use 


Training needed 


NO special 
tools for 
teacher 


/ 

+ 


' . 1 . 4 ' 


For single creation just 
few provisions, possible 
to Implement from 
external medium but 
also with difficulties. 


Yes, all of them, system is 
based on www Apache 
server - files through http 
protocol, no streaming, but 
possible to have a link to 
other server with any 
technology 


Yes, all the 
forms 


Yes, supported - file 
formats e.g. 
.wmv,.wav,.swf, 
.rm,.rpm,.asf, audio, 
video, flash, 
multimedia 


AICC 

compatible. 

Graphics, 

moving 

images, 

audio, video 

all 

supported. 


+ 

/ 
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Testing is sufficient, 
database of questions, 
possible to set time 
parameters 


Plenty of provisions: 
selftest, questions types- 
multiple choice, calculating 
ones, automated 
assessment of text; inquiry- 
anonymous answers on 
questions, submission of 
assignments for assessment 


Integrated 
system for 
examining 
by means of 
WWW, central 
database of 
student 
results 


Test assessed by 
teacher, self- 
assessment test from 
question types: 
multiple choice, 
answers yes or no, 
true or false, written 
essays; provision to 
set minimal exam 
requirement in %; 
results are registered 
in personal 
assessment student 
book, assignments 
for group or all 
covering/universal 
assignments 


Questioning, 
assessment, 
assignments 
must be 
included in 
course. LMS 
stores 
results only. 


+ 




- 


+/- 


+ 


+ 


-/ + 





Explanation: (?) overall, (-) negative impression, (+) positive impression, (-/+) rather negative than positive 
impression, (+/-) rather positive than negative impression, (--) information not provided. 



In house developed systems 

One of the positive points of the self-developed system is that the 
institutions are adjusting their LMS according to their needs and 
requirements. This refers to the course creation as well where certain 
features are constantly being updated in order to meet the needs of 
course developers. One respondent states: 

"As we were developing the system on our own we were actually all the 
time learning from our mistakes. Each of the stage was applied in 
practice on a certain segment of users in order to improve immediately 
every step. Nowadays the system meets our needs and enables easy 
work for students and teachers. If it was not so, it would be impossible to 
have the system in operation, because our teachers are external 
specialists (mainly university teachers) and they would not accept 
complicated co-operation. They supply the courses with know-how (e.g. 
Maths, Management, Economics, Law, Accounting etc.) and our employee 
co-operate with the teacher as a scenarist. It means that we make the 
most of suggestions and corrections and after that the teacher judges the 

content correctness and efficiency. " 

( 

In this sense after familiarisation with the system the use is not 
complicated in all three cases. 

The question on structure and didactic flexibility brought 3 different 
answers as "yes LMS permits didactic flexibility", "A fixed structure of 
course support was given" and "the didactic flexibility is planned for the 
future". 



Graphical material are usually an Internal part of every course, alike 
audio and video, moving images are supported. 



The LMSs provide a vari ety of options for g uestionlnq.-assessment-and 
assignments^as self-testing questions, direct posting of questions to the 
tutors, posting of questions to other users (common discussion), direct 
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posting of administrative and technical questions, subject related FAQs, 
group assignments, tests according to the nature and character of a 
certain course, the assessment Is an Internal part of LMS, Is made public, 
there Is an automated submission of ready assignments. 



o student support tools 

We were looking for answers on following questions: 



o Interactivity possibilities. What provision does the LMS make for 
student interaction? 

o Online student to student communication (synchronous and 
asynchronous). What facilities does the LMS provide for student 
communication to 'other students and how successful Is it? Is both 
synchronous and asynchronous communication between students 
supported? 

o Online student to tutor/institution communication 
(synchronous and asynchronous). What facilities does the LMS 
provide for student communication to the tutor ion to the 
Institution's administration and how successful is it? Is both 
synchronous and asynchronous communication supported? Are 
these support services available 24 hours a day? 
o Resources, library, references. What facilities does the LMS 
provide for student acquisition of resources required by the course, 
especially library resources and references to required readings? 
o Feedback on work and assignments. What is the quality of 
provision of feedback to students on their work and assignments? 



3 TUTOR2000 



The interconnection among the students as well as communication as 
such is not directly available within the system, it is necessary to use 
external facilities. It was said that this absence of student to 
student/student to tutor. Institution (synchronous and asynchronous) 
communication should have been removed in a short time. (As we have 
received the latest news, it has been arranged already). Nevertheless 
Interviewees used to apply and are used to applying communications 
tools / software in Intranet or Internet such as NetMeetIng, MSN 
Messenger, e-mail. These services are available 24 hours a day. 

Here is one comment: 

"About the asynchronous communication it is said by the authors, that it 
is fully supported by Tutor2000. The truth is that it is necessary to install 
News server for the asynchronous communication. After this installation 
TutorlOOO is able to create discussion groups on various topics (normally 
on the topic of the course)." 
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Resources, library, references, important readings are normally accessible 
through www links to the specific pages as to the University library or to 
the websites of University departments or internal materials of the 
company as well as a system of internal regulations/directives is used. 

With regard to feedback on work and assignments the teachers might use 
the e-mail, the News, synchronous communication or classical telephone 
contact. 

Other standard LMSs 

Table 9. Student support tools - Other standard LMSs 







" 2. Sitfdem 








tearniOgSpace 


WebCT ^ 


' <>m 


Intraleam 


Asper» 


: 

» . . . . 


SlT: 


Virtual classrooms 
(discussions on certain 
topics) for students 


e-mail, group 
discussions, chat 


Interactive "E-lab" 
tool based on 
Flash technology. 


e-mail, chat, 
remarks addition 
into discussion 


Forums, 

results 

review 




- 


and teachers as well 




works as simple 
simulator of 
computer network 
laboratory 


group 


1 


+ 


'O' 


Only asynchronous 


asynchronous 


Only 


Yes, both: e-mail. 


Forums, No 






communication 


communication - 


asynchronous 


chat, remarks 


synchronous 




1 




e-mail, 

discussion group 
synchronous 
comm. - chat, 
graphic tables 


communication 


addition into 
discussion group 


features are 
available 
and required 


+ 

/ 


; i;;: 


Asynchronous for 24 
hours a day: 2 forms: 
l.virtual classroom, 2. 
Student placing of 
assignments (can be 
private-student/teacher 
or public) 


asynchronous 
comrnunication - 
e-mail, 

discussion group 
synchronous 
comm. - chat, 
graphic tables; 
teacher present 
1 hour a day 


Not available 


Same as 2.2, each 
student has e-mail 
account in his 
profile; 

synchronous com. - 
chat only during 
consultations time; 
asynchronous com. 
24 hours a day 


Forums, No 
synchronous 
features are 
available 
and required 


+ 

/ 


2.4: 

•i"!: 


MediaCenter 
(distribution of all 
electronic resources) is 
Internal part of system, 
links to websites 


Various materials can 
be incorporated in 
form of WWW sites, 
ppt presentations 


Only WWW links 


Web links, 
references 


Resources 
included in 
course. No 
special tools 
for resource 
library 

management 


+ 

/ 


2;5^ 


Very good feedback 


Autotest- immediate 


Student can look 


Students results in 


On-line 






system, adjustment of 


feedback; results not 


' at the test results 


overall overview of 


result 




i; 


tasks status as ' In 


registered; 


and correct 


results, solvable 


reports / self 




: ; : 


development' 'sent 


assignments- student 


answers of test 


assignments to be 


tests for 




*. ; ' , 

iiwi 


for comments' 'sent 
for assessment' . 
Solutions to be 
private/public 


get back number of 
points, and a written 
comment by teacher 


questions 


sent as attachment, 
submission deadline 
for assignments can 
be put in students 
calendar 


j students 


+ 




+ 


.+ 


-/ + 


+ 


+ /- 





Explanation: (?) overall, (-) negative innpression, (+) positive innpression, (-/+) rather negative than positive 





impression, (+/-) rather positive than negative impression, (— ) information not provided 
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In house developed systems 



Among interactive possibilities, following provisions were mentioned: 
group assignments, common discussions, instant messaging. 

Both synchronous and asynchronous communication in forms of chat, 
discussion groups, e-mail was already available during the time of 
conducting the interviews or being prepared to be ready in a short time. 
Since the server is usually on all the time, the asynchronous and the 
synchronous communication is available also 24 hours a day. By one 
respondent the synchronous communication is available 10-hour day. In 
one case the connection between student and tutor is provided so far only 
on asynchronous basis. 

Required readings are available online, the lecturer has the possibility to 
make his lectures' materials public in the form of PDF documents on the 
web or to put links to other websites. Students have always access to the 
^ libraries of participating institutions. 

Concerning feedback on work and assignments in one case the feedback 
on self-testing questions is pre-programmed. Other LMS allows to add a 
written comment to the assessment in points. 



o Tutor support tools 

We were looking for answers on following questions: 

o Tracking students - database questions. How user friendly is 
the LMS for tutors wishing to track their group(s) of students and 
retrieve data from the student database? 

o Group management tools. What facilities are provided by the 
LMS to the tutors for managing their group(s) of students? 

o Preparation of questions and assignments by tutor. How 
successful is the LMS in providing tutors with user friendly and 
didactically successful tools for the design of student questions and 
assignments? 

o Course planning for students (monitoring pace). What tools 
are provided by the LMS to tutors to enable them to monitor and 
plan student progress? 

o User-friendly administrative systems between tutor and 
institution. What provision does the LMS make for successful tutor 
to institution communication? 
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TUTOR2000 



The functionality of tracking the students and management facilities for 
the tutor seems to work well and users have positive experience with 
that. The LMS enables the teacher to see his subjects, students, who are 
attending his course as well as all materials that are linked/needed for 
these courses of this teacher. The LMS makes notes of all interactions of 
the student with the electronic course, that means there is a complete 
study protocol available and the tutor has overview about all student 
activities. The teacher can follow the students progress, make conditions 
allowing them to continue in the studies after successful passing the 
tests. And again the teacher has the possibility to use the NetMeeting, 
News (discussion groups), e-mail, text news in order to communicate 
with the institution. One of the institutions was just solving the 
link/connection to their school information system and the other one was 
preparing a planning calendar. 

Some criticism: 

'The authors say, that TutorZOOO allows complete overview about 
student's progress in the study, about his activities and reached tests 
results. The truth is that the system has these possibilities, but in our 
installation some of the things do not function correctly. " 

Here is another perspective of the teacher described as: 

'The current version of the system does not provide any support tools for 
teachers - the position of the teacher does not exist in our company. 
There are only people who guarantee for the content of the electronic 
courses and these are employees of expert departments. The whole 
management and organisation of the study is governed by the education 
department. " 



Other standard LMSs 
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1 . LeamfngSpace 


w^bcr 


. cln 


intraleam 




Ml 


liiliilii 


Good, student 
contributions easy 
accessible, sorted 
according to topic, 
students, dates... 


By smaller number of 
students is good 
single server version; 
by bigger number is 
better Campus 
version, where link to 
external databases is 
built in. Possible to 
be linked to student 
database. 


Tests results are 
accessible by the 
teacher 


Work/ communicate 
with group; select 
tasks, assignments; 
control student's 
procedure & 
progress 


LMS follows 
start date, end 
date, total time 
spent in course, 
self test results 
review 


+ 




Group defined at 
beginning, after 
-theirvirtual 
classroom is 
1 created, where 


Discussions, 
advanced statistics 
evaluate student 
work, access to study 
materials is 


Opening the tests, 
to follow the 
statistics, 
cancellation of 
unsuccessful tests 


Tools for 


Forum, mall 




_rTianagernenL-or 

groups - group 
tasks, chat, 
discussion forum 




+ 
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Student profiles, 
portfolio are In. 


monitored 












Tasks organised 
by/in special 
modules 


Developed out of LMS 


Assignments are 
prepared in 
advanced by 
system provider 


Advanced system 
for self-assessment 
tests creation, 
patterns for 
working out of 
assignments 


Testes/ 

questions must 
be designed in 
courses, LMS 
saves, 
summarises 
results 


+ 

/ 




Course structure 
can be displayed 
in schedule or 
calendar with 
student duties, 
actlvities-time 
unlimited, time- 
limited, fulfilled in 
time Interval 


Syllabus, calendar, 
assessment, 
assignments, possible 
to adjust accessibility 
to single course 
elements, time 
monitoring of 
websites access 


Student progress 
is determined by 
chapters, for 
which there are 
tests opened 


Possible to limit 
student's access for 
specific time, follow 
his progress in 
course within that 
specific time period, 
put tasks dates into 
the student's 
calendar 


LMS follows 
start date, end 
date, total time 
spent in course, 
self test results 
review 


+ 


iiii 


Only in way of 
discussion 
contributions on 
tutor, no external 
communication 
out of course 


None, Single-server 
version has no 
external line-up 


Management of 
course is directly 
done by teacher, 
whole interaction 
with LMS works 
through www 
interface 


In the form of e- 
mail, face to face 
meetings 


No tools 


/ 

+ 


: 


+ 


±1j: 


+ 


+ 


+Zz 





Explanation: (?) overall, (-) negative impression, (+) positive impression, (-/+) rather negative than positive 
impression, (+/-) rather positive than negative impression, (— ) information not provided 



In house developed systems 

Replies within this section showed global satisfaction, where e.g. one LMS 
provides list of all students (not yet with a search functionality, but 
planned to the future), enables the tutors to view all users 'activities 
undertaken online, statistics of all users (movement through the course, 
self-testing questions opened and answered, communication). 

Design of student questions and assignments is supported and considered 
as successful, simplifying the teacher's work in the following way: 

"The system provides templates for preparing the course materials where 
the self- testing questions are an integral part. The same applies to the 
tests and the assignments which are than uploaded to the system. " 

And at the same time tutor communicates with the institution as 
described: 

"LMS allows for direct online communication between tutor and 
institution, instant messaging, mailbox reminder, communication history." 



o Administration 



We were looking for answers on following questions: 



o Enroiment procedures and fee paying. What facilities does the 
LMS provide for; student enrolments, course allocations and 
payment of fees? ; 
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o Passwords and security. How successfully does the LMS handle 
student access to the system and the security of all student 
interactions with the system? 

o Student records database. How successful is the system's 
student database, especially for data storage and data retrieval. 

o Examination and certification records. What structures are 
provided for recording of data and results leading to examination 
and certification? 

o Course, ciass and tutors database. What facilities are provided 
for administration of courses, classes and tutors? 

5 TUTOR2000 



The student registration is done by administrator directly in the system. 
The course is allocated also directly by means of the system and this is 
done either by the administrator again or by a teacher, if he has the right 
to do so. One of the interviewees stated that they were just preparing an 
interconnection between the LMS and a database "Student", where 
students can sign in obligatory and optional courses/subjects. 

The password and security issue has invoked quite negative reactions as 
e.g.: 



"I am not that satisfied with passwords and security. It has happened to 
me more times already^ that after correct sign out, I could sign in without 
system's request for my password. By the sign - in the system has a 
option "to save the password" and by the use of the same PC by more 
students (and by their carelessness) it can happen, that the next student 
can get into the account of the previous student. The system does not 
have protected courses (they are launched / opened in the new window 
and their address can be obtained in more ways)." 



On the question of student record database Tutor2000 was considered as 
having a disadvantage that student can change all their data in the 
system as name, surname, address, date of birth..., what can cause 
chaos in the database. ; 



By larger companies it is usually the case that an Accounting system (AS) 
already exists by the time of implementation of e-learning systems. It 
might be e.g. SAP, which covers all finance, controlling, human 
resources... of the firm. That means that LMS is coming as a separate, 
independent tool and is usually implemented parallel to the Accounting 
system. So far, the integration between the LMS and AS systems does 
not seem to attract much attention. This same finding appears in a 
comparative study on Online Education Systems in Scandinavian and 
Australian Universities written by Morten Plate Paulsen 
(http://www.nettskolen.coni/in enalish/webedusite/ ') where the author develops this 
ide a further "One mav however-prediGt-that-this-inteQration~wiiri3e more 
and more important as online education generates more income for the 
institutions." 
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By the Czech Insurance Company it Is the case that in order not to 
duplicate databases, the data about the students and their course 
attendance are transferred into the SAP and Tutor2000 is really only a 
connection of LMS and Student database in other words LMS and 
Students Management System. 

Other databases as for courses, classes or tutor administration are not in 
use by any of the respondents. 

The question 'What structures are provided for recording of data and 
results leading to examination and certification?' brought the explanation 
that the system notes different statistics as: 

- Activities of group of users: Number of allocated courses, finished 
successfully, finished without success, not finished, not started at all 

- Activities of the user: Name of the chapter, status (started, not 
started), max. score, required score,... 

- Activities of the users according to the categories (subjects)... 
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Other standard LMSs 



Table 11. Administration - Other standard LMSs 















ggiiiiighiiiiibe 


i " i ' 'WebCT' . 


mi'. 




A$pier» . 


i ^ 


is 


LMS linked to 
IS of Uni./ 
there is 
complete 
student 
agenda. 
Payment 
system not 
available 


No tools for economic agenda; 
student course allocation/ 
student enrolment done by 
system administrator or teacher 
or student 


WWW 

interface 

1 


Each course has 
stated price/ 
students can 
register into course/ 
payment done 
externally/ not 
through LMS 


All needed 
processes & tools- 
Course selection/ 
enrolment/ 
approving/ fee 
calculation/ fee 
allocation 


+ 

/ 


•• i • 

; *. .5 


After course 
enrolment 
student 
account is 
installed/ 
teacher places 
students into 
courses 


User name/ passwords 


Use of 
HTTPS 
protocol 


Unique log in name 
and password 
according to code 
after payment/ this 
information is sent 
by e-mail/ owner 
can modify this 
data 


Log name & 
password 


+ 


4.3; 


satisfactory 


Not 

good 


Our single-server 
version uses internal 
database-possible to 
make fije backup/ 
updates/ but no data 
corrections by external 
programme; Campus 
version enables to 
connect data on 
outside SQL database 


Easy, 

powerful/ 

intuitive/ 

no 

problem 


Student database/ 
all related data can 
be ex-/imported/ 
archived 


No problem/ a 
large and stabile 
database 
structure 


+ 

/ 


4.4 


Result 

portfolio from 
tests, 

assignments 


Not 

good 


Number of reached test 
points; time, when task 
was finished; 
comments of teachers 




Records of results 
contain currently 
attended courses/ 
number of taken 
tests/ examS/ their 
results/ number of 
reached certificates 
of student 


Very good data 
Structure- 
Students + skill + 
position 

description + by 
course delivered 
competencies 
data structures 


+ 

/ 




Not solved by 
LMS 




Administration of 
teachers' accounts is 
common with 
students' accounts 
administration 




It is responsibility 
of system manage^ 
he defines teacherS/ 
allocates courses 


On-line & FTP 
courses 

manageable by 
LMS/ courses and 
curricula tools 


/ 

+ 




±/z 


-/ + 


+/- 


+ 


+ 





Explanation: (?) 



Cl ail/ ^ j iicyativc ii i i|ji C99iui 1/ J pwoiLivc II I ipr C99IUI 1/ \/iJ idLllcr nc^dLlVe Lildn pOaluVc 

Impression/ (+/-) rather positive than negative Impression/ (— ) Information not provided 

In house developed systems 



By our 3 questioned individuals 3 different answers came out on the issue 
of enrolment procedures and fee paying. One applies classical bank 
account transfer that means it is not governed by LMS. By AINova the 
payment of fees is just planned to be incorporated into the system. But 
this system enables already the learners to enrol online, the course 
material can be accessed online, downloaded or printed. 

Each student obtains his password allowing to access only certain parts of 

the materials depending on student enrolment, time, group work... 
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Work with the system's student database was qualified as successful, or 
in other words as not having significant problem and always available 
also to the students. 

Examination records are normally stored in so-called learners account or 
in a database, which is an internal part of the system. The final score 
from the assessment is a base for an exam. 

There is usually a whole database for courses, classes and tutors posted 
online. 



o Technology 

We were looking for answers on following questions: 

o Server - hardware and software options. What is the quality of 
server hardware and software options? How is the system 
integrated with existing software? 

o Client - hardware and software options. What is the quality of 
client hardware and software options? Does the system permit 
metatagging? 

o Flexibility of didactic structure; updating, adaptability. Is the 

didactic structure flexible or is it determined by the technology? 
How adaptable is the technology to updates and to new technology 
that becomes available to the market? 

o Limitation of size (number of students, courses, tutors). How 
satisfactory is the LMS for handling varying numbers of students, 
courses, tutors? How does it cope with 100, 1000, or 10000 
students and large course databases? 

o Speed of system. How is the speed of the system and student 
satisfaction? How does it cope with downloading courses and high 
bandwidth materials? 



TutorZOOO 



The typical answers were: 

"The system is integrated into MS Windows 2000^ server of the firm Dell^ 
CPU 1 GHz." 



" The system is installed in the Intranet on the server of the company ^ 
(web-farm f load balancing), the database is on SQL cluster. The whole ^ 
system is accessible in the Intranet (HTML and ASP technology)." ^ 

o 

The usual requirements on the client's workstation were described as y 
follows a standard PC (min. Pentium 200MHz, 32MB RAM), Internet g 
Explorer min. 5.5 SP2, Microsoft Virtual Machine, MS Windows 2000. ^ 
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The didactic structure is very flexible, the system is easy to be updated 
and at the same time it is possible to implement also new technologies 
into the system. The speed of the system is on the standard level so- 
called normal display speed of HTML sites, but as it was stated the speed 
always depends on the technical equipment of the client and on the 
Intranet / Internet lines, which are sometimes very slow. 

Considering the limitations of size (number of students, courses, tutors) 
there came always the same kind of reply in the sense: 

"There is no difference between 100 or 1000 users, it is given by the 
quality of the hardware devices. This counts also for the course 
databases. The limits are given by the size of the disk space and by the 
functionality of the SQL database. " 

Other standard LMSs 

Table 12. Technology - Other standard LMSs 
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Hi 


Sil-;- 


Old 


Not high 


LM5 based on 


Server operated 


Server wlth- 


For Microsoft 




•' ’ . 


software/ 


requirements. 


WWW Apache 


by Cisco 


processor IGHz, 


NT/SQL or 




: J. . 

•■'i ^ 


hardware 


no integration 
problems 


server on 
platform WinNT or 
Linux, possible to 
install ordinary 
modules into 
server 


Systems 


512 MB RAM, 40 
GB Hard disk; 
software - 
Server NT 2000, 
MS SQL 
database 


Unix/Oracle 

platform 


1 + 


S*2 


— 


Usual hardware, 


www' browser. 


WWW browser 


Client needs- 


No special 








Java scripts support, Internet link 


with Rash 
Player pug in 


MS Windows 95, 
Macintosh, IE 
5.0, NC 4.75; 
LMS permits 
metatagginq 


requirements, 
only Web 
browser 


32 


513 

1 

' : : 

; ' r*. 


Bad 


LM5 provides fixed group of tools, 
their use in course is flexible 


Structure 
separated from 
technology, 
course materials 
in one form 


Allows updating, 
downloading of 
needed 
software, is 
created by 
modern 

technology - MS 
SQL databases 


No relation 
between LMS 
and didactic 
structure 


+ 

/ 


>5i4i 


Problems 
by massive 
application 


No problem until now 


Without capacity 
problems 


No problems 


No limits 


+ 

/ 


'i^'s" 


Not 


Fast access to 


LMS works on 


Technical core in 


Platform 


No problem with 






problem on 


sites 


standard www 


US is powerful. 


allocated on 


LMS speed. 






LM5 Side 




Apache server - 
the most 
widespread of ait 


limited only by 
Internet lines 


institution's 
server - fast, 
limited only by 
Internet lines 


dependent on 

communication 

infrastructure 


+ 


> 


- 




+ 


+ 


+ 


+ 





eg 



CApiofidLmfi . \'J uvcidii; \ j iicydLivc irnprcbbiuri/ L ' / pobiLivc irnprcbsturi/ \/*j raLncr ncydLiVc Lrldn positive 
impression, (+/-) rather positive than negative impression, (— ) Information not provided 

In house developed systems 

The quality of server - hardware and software was considered as good 
and the integration into the existing software was just planned by one 

” The question number 5.2 is not assessed beq|jse the institutions stated factual data. 
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higher. No other software requirements are necessary for the client. 
Metatagging is supported. 

Concerning flexibility of didactic structure; updating, adaptability the 
LMSs are considered as having no problem other Is planning it for the 
future and the third Interviewee states: 

"The system has been built with the target of flexibility of didactic 
structure and that has been reached. " 

Looking at the numbers of students by all three is this number quite low 
and maybe therefore no problems have occurred until now although they 
might be working already on a limited number. One remarked directly: 

'The system is not adapted for a targe number of students yet, its 
maximum capacity is 100 students. " 

P The speed of the system was defined as satisfactory always determined 
by the line speed of the institution. 

o Price aspects 

We were looking for answers on following questions: 

o Cost of the LMS (Learning Management System). What is the 
cost of the LMS to the institution? 

o Annuai fee. What fees have to be paid annually for the system by 
the institution? 

o Student Enroiment fee (100 students, 1000 students, 10000 
students). How do fees to use the LMS vary when the student 
base is 100 students, 1000 students, 10000 students? Is online 
invoicing available? 

• o Maintenance costs: staff invoived in management, IT 
speciaiists, trainers, etc. What is the maintenance course to the 
institution of the LMS and what staff resources are need to maintain 
it and keep it functioning? 

o Training of teachers and iearners and system users. What 
costs are involved in staff and student training to use the LMS 
system? 

The figures in following tables are sometimes stated in three different 
currencies. The reason is that institutions usually provided this 
information in their national currency. These figures were then converted 
into Euro and USD. In cases when the information was given directly 
either in Euro or USD, it remained the same and was not calculated 
further. 



Tutor2000 



nuc 
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Three from our Tutor2000 users have provided us with the price/cost 
Information, which could be structured as follows: 



Table 13. Price aspects - Tutor2000 





Value 


Total costs: payment to: ^lecorhfy^&ipensesFfor 
serveri, wnH^tatlohs, netwhrk, w&rkers etc... 


3 Million CZK / 98 63© 96 855 

USD^ 


'UMS;. in configuratibh, bf TSO. iitutora. -and 
■students': • • 


6969 USD, ToolBook II included 


;;Annoa[h;tee' : 


0, no fee for already bought system, 
considered as great advantage 


Tees, dependent on ;:$tudent base. (lOP, lOOO, lO. 
ibOpiStbden.ts) . 


There exist amount discounts for 
the student licenses: up to 50 
students - 500C2K/1C4 ^7l6 

USD“ /license, up to 500 students - 
200 C2K/6.6€ ^°/6.5 USD“/license, 
for more than 10,000 students - 80 
C2K 2.6 €'72.6 USD'Vlicense. 


Maintenance costs i- for custorher^s adjustment 


34 USD / hour 


ITrainihg: " 


Annually 


20 000 C2K/657.5 €'7646 USD" 
per teacher 




Gharcfe:fbr. an :bbur by theiflrm . . . 


34 USD / hour 




One day:training.:fbr 20 iteachers- by the 
firm 


30 000 C2K/986 €'7968.5 USD" 



Normally the institutions cannot afford such expensive services therefore 
the run of the system is secured by employees of the faculty within their 
normal working tasks and so there are no further costs required or other 
university employs 5 specialists full time. One is specialised in the LMS 
Tutor 2000, the second one is responsible for the work and training for 
ToolBook II Instructor, the others prepare patterns for the single courses, 
the teachers can choose then. All 5 of them have passed the distance 
course for distance education. 

In the same way the expenses on training are being saved: "The first 
one-day training was provided for teachers by the firm. The following 
trainings are run by the requalified employees from our Centre for 
information technologies within their regular working time/content. " 



Counted with the Czech/Slovak National Bank exchange rate from the 29“’ July, 2002; 1€ = 30.42 
CZK/44.545 SKK. Source: Czech National Bank - www.cnb.cz/en/index.html. Slovak National Bank - 
www.nbs.sk/INDEXA.HTM 

Counted with the Czech/Slovak National Bank exchange rate from the 29“’ July, 2002; lUSD = 30.974 



CZK/44.543 SKK. Source: Czech National Bank - www.cnb.cz/en/index.html. Slovak National Bank • 
www.nbs.sk/INDEXA.HTM 
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other standard LMSs 



Table 14. Price aspects - Other standard LMSs 
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Amu 


CoStOfLHS 


Fees for 
hardware and 
LMS 

administration, 
LMS was 
obtained from 
education 
support 


Unlimited licence approx. 5,000 
USD 


LMS given for 
free to Local 
Academy of 
Education for 
Programme of 
Cisco 

Networking 

Academy 








Free of charge 


5,000 USD for year 2002, expected 
increase to 7,000 USD 


No fee 


~~ 




Student • 
enfdmeitt 
fee 


Grant per 
student from 
state 


Fixed amount (5,000USD) for 
server licence is not dependent on 
number of students 


No fee 






‘ Mainteiwoce . 
Cosfe 


One technician 
One 

administrator 
Personnel for IS 
Student 


A system 
administrator, a 
spedaiist for 
teacher training for 
course creation, 
both employed part 
time 


Approx. 

100,000 

CZK/3287 

€‘®/3228.5 

USD“per 

year 


LMS is updated 
centrally by firm 
Cisco Systems 


Internal, 

external 

workers; 50,000 
-60,000SKK/ 
1122-1347 €*“/ 
1122-1347 
USD“ + costs 
for teachers 




Training 


Each teacher 
goes through 2 
trainings:!. 
Tutor of 
distance 
education, 2. 
Operation with 
LMS 


For teachers - preparation for 
ECDL (approx. 3500CZK/ 115 
€^0/113 USD“) + ECDL 
certificate (2500CZK/82 
€^0/81 USD^^) + WebCT 
training (approx. 2500CZK/82 
€^°/81 USD“) X number of 
teachers (approx. 35). 

For students - a guide has 
been created + training within 
the seminars/classes 




System is 
intuitive, only 
40 min. training 
needed 


Training of 
teachers, 
students receive 
guidebook about 
system 
operation in 
electronic form, 
a hotline is at 
disposal 





Explanation: (— ) information not provided 



In house developed systems 



All respondents gave us their financial information. 



Table 15. Price aspects - In house developed systems 



BEST COPY AVAILABLE 





. w ;'Vv ^ A'" " '<• 


i Cost of . IMS . • 


Development cost - aop. 6,000 € 




0 - LMS was developed within a students project 




Investment of 1.1 Million CZK/36160 €^°/35514 USD“ 


! Annual fee 


None-since LMSs are their systems 




only annual maintenance costs for servers, fees etc. about 300,000 
CZK / 9862 €^°/9686 USD“ 


i Malnl^na oce costs i 
-staff 


For 1 programmer and 1 supervisor 
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"From the fnandal reasons we can allow us to employ just a few 
workers (currently 2) therefore the further development stagnates 
at the moment. The number is not satisfactory; an enormous 
personal engagement is required. Selected activities are 
accomplished by external specialists." 




1 person in charge of server run, more people co-operate on 
permanent deveiooment 




Cost for 1-day meeting 



! 

i 

Online invoicing was available in one case, but not used, 
o Conclusion 

In this last part respondents were asked firstly to give their overall 
assessment of their satisfaction with LMS and secondly to specify features 
they would like to see included in the LMS In the future. 

a) Overall evaluation 

Tutor2000 

Three reactions expressed their full satisfaction with the system as one 
completely functional and perspectival, emphasising the advantage that 
the whole operation of the system Is In Czech language. On the contrary 
there were 2 critical comments. One pointing on some system shortages 
in comparison to the LearningSpace used before and the other stressing 
markedly: 

"The system TutorZOOO is a LMS, which is mainly for administration 
agenda. Because of the fact that TutorZOOO cannot create courses, there 
is the ToolBook II as a component part of the delivery. From my point of 
view the system has some problems by single installations by customers. 
In our case some functions are not working properly (statistical evaluation 
/ assessment). The system does not have an authorised access into the 
courses (they are opened in a new window and it is possible to obtain the 
address of the course). The system does not have its own means / tools 
for online communication. Considering these all facts I do not think that 
this system in this phase is appropriate for commercial use. " 

Other standard LMSs 

Institutions very openly judged their systems. The shortages were 
criticised and the advantages considered. 

Of the WebCT users one stated "applicable" the other "it fulfils our 
requirements". 

Aspen was characterised as "well done for internal education of large 
company, where asynchronous and self-study methods are preferred". 
Positives are to be found also in the view on G LN: "the LMS is good, its 
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interface for study as well as for administration is completely based on 
WWW, it is wo rid- wide and stable". 

The look on LearningSpace gives us more details: 

" For the current operation of the system in our education (approx. 100 
students, 20 teachers) the system is usable. By higher numbers I do 
predict some problems. The system has quite satisfactory tools for the 
management of the education, it is worse with the tools for course 
creation. There is number of problems. The database administration 
would be very demanding and ineffective by higher number of students. 
The system does not solve the problem of relation/connection between a 
student and an education institution at all. The whole communication is 
just focused on the course. The communication e.g. with the study 
department and the whole study agenda is led/solved in the IS Student. 
The link between IS Student and LearningSpace is not automated. " 

Intralearn was judged as "a modern platform, constantly improving 
according to the requests of middle (firms and educational centres) and 
commercial users (clients, students). By course creation the system 
allows to accept also older, earlier developed materials in different 
formats. There is no special courseware, which would cost a lot of money 
and also from the user's point of view it is very practical. It is not limited 
by any special requirements on hardware and software from final users. 
Administration and the management have not yet been completely tested 
in our conditions, but we can base on references of foreign firms (as well 
as universities), which use the system." 

In house developed systems 



The LMSs were identified by all as good, meeting their needs and easy to 
use. They state that by the means of LMS the teacher's work is getting 
more structured and organised. One admitted the current size limitation 
what makes the system capable to offer courses just to a smaller number 
of learners and for the system's marketability this is also one of the 
features to be improved in the future. 

b) Features in future LMSs 



Tutor2000 



The possibility to apply more demanding technologies (Streaming 
video, videoconferences etc.) 

To guarantee an authorised access into courses and to secure a 
reliable operation of the system, protected communication in all ways 
Integration of more possibilities for online communication (chat, video, 
sharing, voice communication) 

A planning calendar 

More provisions for incorporation of more formats not only HTML, PPT 
etc. 
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BEST COPY AVAILABLE 



other standard LMSs 



o LearningSpace: 

o Support for creation of education multimedia 
materials 

o Better import from other recourses / environments 
o Synchronous communication 
o Study agenda to be directly included as a internal 
part of LMS or automated link with IS Student 
o More attentive environment as for the student, as 
well as for the teacher 
o Czech localisation 

o The possibility of enrolment and fee payment to 
be within the system 

o WebCT: 

o Better support for team work 
o Technology for streaming 
o A connection to external databases also from 
single-server version of the system 

o GLN: 

o The possibility to integrate self-developed 

materials of the institution as separate additional 
materials to the official ones (by GLN, Cisco 
prepares and delivers study materials as well as 
the entire course in advance already) 
o possibility to link own student database to central 
student database 

o Intralearn: 

o More advanced assessment system of reports 
linked to statistics 

o Functional improvement of dictionary of terms 

o Aspen: 

o Tools for vertical structured courses management 
(courses with more levels of modules, chapters 
and sections) 



« 




In house developed systems 



• Better user support, better multimedia support 

• Interconnection of all created modules of the system 

Key findings 



imc 



The trust in e-learning is not yet wide spread. General public opinion 
about online education is not always positive. The institutions are 
many times viewed as the ones implementing strange "things". In 
most cases online education is used as a help and addition to the 
traditional face-to-face education. However there are individual 
experiments and, as one institution stated, they would like to adjust 
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their LMS in the way that it enabies them to offer paid courses as a 
kind of iifeiong education to the pubiic. 

• The institutions are facing significant financiai probiems, are afraid of 
announced charge increases of the LMS. The system deveiopment 
stagnates because Of this iack. Faster impiementation of the 
eEurope+ targets might have been reached if this impiementation 
wouid not be iargeiy coming from nationai budgets in the candidate 
countries. 

• In the Czech repubiic recentiy a nice co-operation in a project of 
buiiding a virtuai university has started among 3 universities (SBA in 
Karvina, University of Ostrava and FE). 

• 9 from 14 interviewed institutions have not been using LMS ionger 
then one year. That shows the very eariy start but the resuits after 
such a short time are aiready visibie. The acceieration of this 

^ deveiopment must be considered. 

• Institutions are siowiy converting to the nationai LMS vendors, since 
the systems are in their mother tongue and want to be active in iarger 
market and offer services to the generai pubiic. 

• There were cases in the study, where a kind of faciiity of a system was 
not avaiiabie (e.g. synchronous communication). It must be 
considered that the institutions are choosing their system and its 
functionaiities according to their future pianned activities and 
requirements. That means that in spite of the fact that the system 
seems to have a shortage it is actuaiiy not the case, because the 
system in that specific form is the most suitable and fully satisfactory 
for the institution. 
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stand Ready? 



Emerging e-learning standards in a pedagogical perspective. 

By Gro-Anett Olsen 

Even though the e-learning industry is a young one, it is said to be in 
need of common standards to ensure further growth. Today half a dozen 
organisations are working to create standards for the industry, and their 
main goal is to make the e-learning technology more flexible and user 
friendly. The standardisation issues have been given tremendous 
attention the last two years, and there exists quite a lot of optimism of 
standards impact on the future of e-learning. 

As with all kinds of standardisation work, you have to let go of 
something. In order to reach the goal of flexibility, you must let go of the 
different variations that exist. So what must be "sacrificed" when 
standards get implemented? This article will look at why we need ^ 
standards, and raise questions of possible consequences that today's 
upcoming standards could have in a pedagogical perspective if variation 
gets limited. 

Key specification players 

"To describe the e-learning standard arena as confusing and arcane would 
be an understatement (...)" Barron (2000:2). 

Standards for e-learning is not an easy field to understand, and you could 
hardly say that there is common consensus about standards today. 
Further, it is important to be aware that we actually don't have any 
standards for e-learning today. Instead there are organisations with 
common interests and arguments that develop specifications for e- 
learning (Singh 2000). Examples of such organisations are IMS, AICC, 
ADLNet, IEEE, CLEO, WC3, ISTO. The first four are the most prominent ^ 
today. Lets have a short look at how they describe their interests. 

IMS - Instructional Management Systems Project 

rhtto: //WWW. imsDroiect.org') 

IMS Global Learning Consortium, Inc. (IMS) is developing and promoting 
open specifications for facilitating online distributed learning activities 
such as locating and using educational content, tracking learner progress, 
reporting learner performance, and exchanging student records between 
administrative systems. 

IMS has two key goals: 

• Defining the technical specifications for interoperability of applications 
and services in distributed learning, and 

• Supporting the incorporation of the IMS specifications into products 
and services worldwide. IMS endeavours to promote the widespread 
adoption of specifications that will allow distributed learning 
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environments and content from multiple authors to work together (in 
technical parlance, "interoperate"). 

AICC - Aviation Industry CBT Committe (http://www.aicc.org) 

The Aviation Industry CBT (Computer-Based Training) Committee (AICC) 
is an international association of technology-based training professionals. 
The AICC develops guidelines for the aviation industry in the 
development, delivery, and evaluation of CBT and related training 
technologies. The objectives of the AICC are as follows: 

• Assist aeroplane operators in development of guidelines, which 
promote the economic and effective implementation of computer- 
based training (CBT). 

• Develop guidelines to enable interoperability. 

• Provide an open forum for the discussion of CBT (and other) training 
technologies. 



ADLNet - Advanced Distributed Learning Initiative 

Shareable Courseware Object Reference Model (SCORM) 
fhttp://www.adlnet.orQj 

The US Department of Defense and the White House Office of Science 
and Technology Policy launched the Avanced Distributed Learning 
Initiative in 1997. The purpose of the ADL initiative is to ensure access to 
high-quality education and training materials that can be tailored to 
Individual learner needs and made available whenever and wherever they 
are required. 

ADL released SCORM (sharable content object reference model). SCORM 
is designed to meet the US Department of Defense's requirements for 
web-based learning content, supporting content reusability, accessibility, 
durability and interoperability. 

SCORM gives specifications for representing course structures (in order to 
move courses from one server/LMS to another), specifications relating to 
the run-time environment, a content launch specification and a 
specification for creating meta-data records for courses, content and raw 
media elements. 



IEEE - Institute of Electrical and Electronics Engineers 
LTSC: Learning Technology Standards Committee 

rhttp://www.ltsc.ieee.orgj 

The mission of IEEE LTSC working groups is to develop technical 
Standards, Recommended Practices, and Guides for software components, 
tools, technologies and design methods that facilitate the development, 
deployment, maintenance and interoperation of computer 

implementations of education and training components and systems. 
LTSC has been chartered by the IEEE Computer Society Standards 
Activity Board. Many of the standards developed by LTSC will be 
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advanced as international standards by ISO/IEC JTC1/SC36 - Information 
Technology for Learning, Education, and Training. 

IEEE cover topics including learning object metadata, student profiles, 
course sequencing, computer managed instruction, competency 
definitions, localization, and content packaging. 



Technological standards for e-learning. Why? 

"E-learning standards are the vehicle that will bring flexibility to content 
and infrastructure solutions" (Singh 2000:1). 

Standards for e-learning will give us new and improved ways of training 
and education in both an individual and organisational perspective 
(Wagner 2000). An important aspect of e-learning is that it depends upon 
technology for implementation. New and improved information 
technologies like databases, learning management systems (LMS), 
learning content management systems (LCMS), search engines etc. are 
giving new possibilities for storing, retrieving and reusing information 
objects across systems, time and geography. 

The standardisation initiatives focus on how to make e-learning even 
more flexible, through making the different new technologies more 
compatible with each other. 

Even if different standardisation initiatives focus on different issues for 
standardisation, there is a common opinion about content portability, 
granularity and interoperability. In Singh's White Paper "Demystifying 
eLearning Standards" he explains these as: 

"Content portability - When content has been separated from proprietary 
delivery systems, the organization can consolidate, organize and track their 
eLearning initiatives in the LMS of their choice. Because this is true for both 
third-party custom content, corporations will have greater flexibility and lower 
switching costs. 

Granularity - The new specifications supports the learning object methodology, 
allowing for smaller and more timely units of information. Learning objects adds 
"just enough" to "just-in-time" learning. 

Interoperability - Application interoperability starts where different eLearning 
applications can share content and tracking data. But even more exciting, these 
specifications open up the possibility for different types of applications to swap 
and access content." (Singh 2000:4). 

For the young e-learning industry to develop further, industry standards 
that ensure this kind of flexibility must be developed, established and 
accepted. 
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Effective learning solutions: LMS, e-learning and standards 

The initiative for establishing standards comes mainly from specific needs 
within large organisations/enterprises that have tradition for using 
technology in learning/training, and that need effective learning solutions 
to meet their challenges. These organisations handles a large amount of 
training to a large number of employees, and they need to do it as 
efficient and manageable as possible. 

The US Department for Defense and the aviation industry has taken the 
initiative in two of the most prominent standardisation workings today: 
SCORM and AICC. These organisations have a vision of using new 
technology to make learning and training more effective and tailored, 
both for the individual and the organisation. Further, they wish to 
stimulate the market in order to increase variation and quality in training 
and educational offerings. ADL writes this about the SCORM initiative: 

"The Department of Defense (DoD) and the White House Office of 
Science and Technology Policy (OSTP) launched the Advanced 
Distributed Learning (ADL) initiative in November 1997. The 
purpose of the ADL initiative is to ensure access to high-quality 
education, training and decision aiding ("mentoring") materials that 
can be tailored to individual learner needs and made available 
whenever and wherever they are required. 

This initiative is designed to accelerate large-scale development of 
dynamic and cost-effective learning software and to stimulate a 
vigorous market for these products in order to meet the education 
and training needs of defense and industry in the 21st century. 
ADL is developing a common technical framework for computer and 
Web-based learning that will foster the creation of reusable learning 
content as "instructional objects." (ADL 2001: 1-11). 

An important part of the standardisation initiatives for e-learning are 
learning management systems (LMS). This is essential for an e-learning 
framework in order to distribute, administrate, navigate, document, 
report and manage e-learning for users and enterprises with different 
needs and strategies. The standards that eventually will be established 
wiil not only apply for e-learning content but also for learning 
management systems in the market. 

E-learning is dependent on an LMS (or some kind of management 
system) for distribution and administration, but from a standardisation 
perspective e-learning should be independent of different types of LMS. 
E-learning should work without problems across different technological 
platforms. This will make the choice of learning technology, if it is for e- 
learning or LMS, independent of vendor. Hopefully, this will open the 
market for more vendors and give a higher degree of variation in learning 
technology and e-learning products. 
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As we see, the upcoming standards has an organisational dimension, 
where the Intention is to manage large volumes of content and users In 
order to get more effective learning solutions without having to deal with 
technology problems. For large organisations like the US defence Industry 
and the aviation Industry, the standards will have an enormous effect on 
internal training for both Implementation and administration. Anyway, the 
intention of the standardisation initiatives is not administration, but as 
ADL puts it: "(...) to ensure access to high-quality education, training and 
decision aiding ("mentoring") materials that can be tailored to individual 
learner needs and made available whenever and wherever they are 
required". (ADL 2001:1-11). 



Technological standards and pedagogy 

"It's a siren song few training professionals can resist: e-learning content 
that's free of proprietary confines and manageable as discrete building 
blocks that can be mass-customized for learners" (Barron 2000:1). 

The standardisation initiatives have, as their primary goal, to Increase 
access to learning material, training and education of high quality that 
can be tailored to actual needs. Standards do not Imply anything directly 
about what kind of pedagogical approach to take when creating e- 
learning content, or what type of functionality a LMS should have. But 
most likely standards will have implications for how content and an LMS 
should be developed and work. To take this one step further, it could be 
argued that possible implications would set premises for what you could 
expect of high quality content in different competency areas. 



Learning objects model 

Sometimes you get the impression that working with industry standards 
for e-learning is just about technology. In articles and speeches on the 
subject you get to know that technological standards, and metadata 
specified by theses standards, in principle doesn't have anything to do 
with the content. The metadata is only a reflection of the actual content 
(REN 2002). 

To retrieve, reuse and blend different learning objects are some of the 
main goals of the standardisation work. A presupposition to realise this 
goal is to structure and tag the content according to a standard. For 
example the content must be structured in such a way that each 
"module" is an independent unit, which is expressed through a set of 
metadata. This opens up the possibility of mixing different e-learning 
units and put them together for new purposes and in new learning tracks. 

Today's prominent standardisation work is based on a model called the 
learning object model or object-oriented design (Barron 2000, Koper 
2001, Downes 2001, Longmire 2000). The fundamental idea behind this 
model is that learning content can be split up and put back together in 
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new learning tracks/courses in the same way you play with blocks of 
LEGO™. 

This model belongs to a systematic and prescriptive approach to 
pedagogical design. The approach has its parents in behaviouristic 
psychology and systems engineering (Molenda 1997). According to Gress 
& Purpel (1988) most of the American research and literature on 
pedagogical design comes from this approach. Theoreticians that are 
known for work in this field are among others Franklin Bobbit, Ralph W. 
Tyler, Taba, Gagne & Briggs, Weinstein & Fannini, Robinson, Ross 8i 
White, Dick & Carey. j 

The systematic approach to the pedagogical field is one among several 
approaches. Other approaches go under names and terms such as 
dialectical, dynamic, evolutional, hermeneutical, and constructivistic. 
Common for all these approaches to pedagogical design is that they are 
critical to a systematic approach and the use of prescriptive and linear 
models on all kinds of competency areas. It is believed that there are 
areas of competence where prescriptive and linear models are unsuitable 
for creating useful and effective learning experiences. For more complex 
learning experiences, they are regarded as too static. 

As was said earlier, metadata and the course structure are said to have 
no implications for the content of the course. An important question when 
you look at standards in relation to pedagogy is if structure and content 
can be separated? Could structure and content be treated separately 
without consequences for each other? Could this be a misunderstood 
theoretical approach that would be impossible to implement in reality? 
Will a predefined course structure set important premises for pedagogical 
design of e-learning? 

If you look at these questions in the light of the learning object model, 
the distinction between content and structure are unproblematic. In such 
a pedagogical approach the structure would be predefined independent of 
the content. But if you look at the question in the light of, for instance, a 
dialectical approach, content and structure would evolve in interaction 
with each other. The structure could not be predefined because it is 
decided on in relation to the content. 

Rob Koper from The Open University of the Netherlands, is one of few 
who work with issues of standardisation in a pedagogical perspective. He 
has taken another pedagogical approach to the standardisation work 
other than the learning object model, and is contributing constructively to 
include another pedagogical perspective in the standards. In his article 
"Modelling units of study from a pedagogical perspective" (2001) he 
directs a critical question to the concept of learning in the learning object 
model. As it has been pointed out earlier in this article, the learning 
object model takes for granted that you can split and put together 
learning units to create new learning tracks. Koper (2001) means that 
such an approach is too. simple, especially when more complex learning 
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processes are needed. In his article he takes on a situated learning 
approach described by Lave & Wenger (1991). According to this 
approach's learning concept, learning is an activity that takes place in a 
sociocultural context. Learning will thus depend on this context. 

In Kopers (2001) analysis of the way the learning object model were used 
In the standardisation work, he found that how these learning objects 
would be used In context, was not accounted for. In other words he found 
a lack a framework. Koper writes: " The learning object model expresses 
a common overall structure of objects within the context of a unit of 
study, but does not provide a model to express the semantic relationship 
between the different types of objects in the context of use in an 
educational setting" (Koper 2001:5). 

Kopers intention is not to say that the standardisation initiatives only will 
do for simple learning processes. What he tries to accomplish is to set 
focus on the need to develop a meta-model that includes the description 
of the learning objects in a semantic context. 



The relationship between technology and pedagogy 

There is good reason to question the idea that a single model for 
pedagogical design is enough to be used in all areas of competence. If 
the technological standards limit variation in pedagogical approaches, the 
use of e-learning for training and education will also be limited. 

A relevant question in this matter is where possible limitations exist. Is 
ity in the technology or in the chosen pedagogical approach? Because 
people with a technological background dominate the standardisation 
work there is reason to ask, without the intention of being rude, if they 
have the necessary knowledge about pedagogical design and learning to 
include different pedagogical approaches in the work. If they don't know 
the field of pedagogy they would probably use their own models and 
methods when designing e-learning, which in this case is similar to object 
oriented design and the notion of learning objects. Fuhua Lin, who 
criticise the way learning objects is used and written about, expresses it 
like this: "The LO model reinforces the notion that course development 
now needs to follow a systems development life cycle. It is quite clear 
that courses are more than a collection of learning objects" (Lin 2001:2). 

If it is so the that ideas and thoughts of people working with standards 
sets the premises for pedagogical variation and quality in e-learning, 
standardisation work should actualise the relationship between 
technology and pedagogy. Because the technologists today have come far 
in the work with industry standards for e-learning, it is very important 
that pedagogues start to engage in the work. This is order to broaden the 
perspectives, and to get the necessary acceptance in the pedagogical 
communities, that eventually will be using these standards. 
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Flexibility or rigidity in e~learning? 

Standards for e-learning technology are created in order to make 
technology and market more flexible and open. This is indisputable a 
good intention. But will flexibility also be the case for pedagogy and 
learning? Because today's standardisation initiatives base their work on a 
single model for pedagogical design, the consequence could possible be 
rigidity in pedagogy and learning. If this is the case, and it needs to be 
looked at in further detail, it gives rise to a new set of questions. One of 
them would be if standards make e-learning less suitable for different 
learning experiences, hence restricts a wide use and acceptance of e- 
learning. 

It has not been the intention in this article to be negative towards the 
standardisation initiatives. The intention is to look at the complexity of 
the issue, and possible consequences today's standardisation work could 
have for pedagogy and learning. Hopefully, this article can contribute to 
an interesting debate in both technological and pedagogical communities 
of whom all are interested in a successful future for the e-learning 
industry. 
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